Recording medium on which 3D video is recorded, playback apparatus for playing back 3D video, and system LSI

ABSTRACT

A left view graphics stream and a right view graphics stream recorded on a recording medium each include one or more display sets, and the display sets are data groups used for displaying one screen worth of one or more graphics objects. The one or more display sets included in the left view graphics stream are in one-to-one correspondence with the one or more display sets included in the right view graphics stream, and an identical playback time for the video stream on the playback time axis is set in each of the corresponding display sets. Each display set includes state information indicating whether the data is all data necessary for displaying one screen worth of one or more graphics objects, or is a difference from the immediately previous display set, and the state information included in corresponding display sets indicate identical content.

This application is a Divisional of U.S. application Ser. No.12/569,016, filed Sep. 29, 2009, which claims benefit of provisionalU.S. application Ser. No. 61/101,329, filed Sep. 30, 2008.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to stereoscopic view of graphics whenusing a 2-graphics decoder structure in a playback apparatus.

(2) Description of the Related Art

In recent years, distribution has rapidly increased of Blu-ray Discs(hereinafter referred to as “BD”) compliant with a next-generationoptical disk standard. A playback apparatus compatible with the BD-ROMstandard enables enjoying video that has a high degree of realism byoutputting a high-quality video stream in which subtitles and graphicshave been superimposed on a display apparatus such as a displayconnected to the playback apparatus (for example, see patent document1).

Meanwhile, as a technical trend of display apparatuses, not only flatimages, but also stereoscopic view displays have begun to bedistributed, which enable enjoying videos that seem to leap out at theviewer. Although there are various methods for stereoscopic displays, asa basic principle, a stereoscopic video is artificially created byimplementing a system of showing different pictures to the left andright eyes as a left view and a right view, using the difference betweenthe two eyes.

SUMMARY OF THE INVENTION

An issue arises of how to perform stereoscopic display of graphics to becomposited with video and played back. Here, when the graphics aresubtitles or a menu, the subtitle or menu can be caused to appear as ifprojecting out from the screen by shifting the coordinates of the imagedata making up the non-compressed graphics object according to the leftview playback time and the right view playback time, respectively. Inthis coordinate shift, the graphics merely appear in front of the screenin a flat state, and do not appear stereoscopic themselves.

In movie works of recent years, there is an example of realizinganimated graphics that meticulously synchronize the characters in themovie work by using a data structure of a Presentation Graphics Streamthat is mainly used for subtitles.

In a stereoscopic view caused by a coordinate shift, when the graphicsrepresent, for example, an animated character, it merely appears that aflat character is in front of the screen, and stereoscopicallyreproducing such features as the hollows in the animated character'sface or the way the nose protrudes is impossible. For this reason, thereis the view that this technology is insufficient for entertainmentpurposes.

In view of this, to make an animated character appear stereoscopically,there is an idea of combining left view graphics data and right viewgraphics data in one graphics stream, and by decoding the graphicsstream, causing realization of stereoscopic view.

However, when using this idea, since the decoding load of the graphicsdecoder is doubled, a clock frequency that is a reference for operationof the graphics decoder must be made higher, thus preventing low productcost.

To avoid making the clock frequency higher, one approach is to make thegraphics data have a 2-stream structure, and make the playback apparatusa 2-decoder structure. In other words, the left view graphics data andthe right view graphics data are built into separate graphics streams,and the playback apparatus is equipped with two graphics decoders. Then,causing the two graphics decoders to process the two graphics streamsenables causing the playback apparatus to decode the left view and rightview graphics data without making the clock frequency too high.

Meanwhile, there is a problem that, when using the 2-decoder structurein the playback apparatus, the stereoscopic view of graphics duringrandom access cannot be ensured. This problem is described specificallyas follows. The left view and the right view graphics streams arestructured respectively to include a plurality of display sets(hereinafter referred to as “DS”), and each DS is a data group to beused for displaying one screen worth of one or more graphics. There area plurality of types of DS. There are DSs that include all datanecessary for displaying one screen of graphics, and DSs that includeonly a difference from the immediately previous DS. A DS that indicatesthe difference from the immediately previous DS cannot independentlyperform display of the entire graphics. Here, since there is adifference in perspective between the image seen from the left view andthe image seen from the right view, even in a case in which all datanecessary for display of one screen of graphics are included, there arecases in which there is a great change from the immediately previous DSin the animated character in one DS, while due to a blind spot, there isnot a great change in the animated character in the other DS. Therefore,it is thought that only including only the difference from theimmediately previous DS is sufficient.

In this case, a problem occurs that, when skipping from these DSs in theplayback apparatus, even if decoding can be performed correctly for oneof the graphics, for the other one, the data necessary for playbackcannot be provided to the decoder and cannot be decoded. As a result,there are cases in which a stereoscopic view of graphics is impossibleduring random access.

The present invention aims to provide a recording medium that ensures astereoscopic view of graphics during random access.

To achieve the above aim, the recording medium that is an embodiment ofthe present invention is a recording medium on which is recorded a videostream, a left view graphics stream and a right view graphics stream,wherein the left view graphics stream and the right view graphics streameach include one or more display sets, each display set is a data groupused for displaying one screen worth of one or more graphics objects,the one or more display sets included in the left view graphics streamare in one-to-one correspondence with the one or more display setsincluded in the right view graphics stream, and an identical playbacktime for the video stream on a playback time axis is set in display setsthat correspond with each other, each display set includes stateinformation indicating that the display set is one of (i) all datanecessary for displaying the one screen worth of one or more graphicsobjects, and (ii) a difference from an immediately previous display set,and identical content is indicated by the state information included indisplay sets that correspond with each other in the left view graphicsstream and the right view graphics stream.

According to the structure described above as means to solve theproblem, state information of display sets of the left view graphicsstream, and state information of display sets of a right view graphicsstream that corresponds to the display set, have identical content. Forthis reason, when the state information in a display set of one graphicsstream indicates that all data necessary for display of one screen worthof graphics are included, the state information in the correspondingdisplay set of the other graphics stream indicates that all datanecessary for displaying the one screen worth of graphics are included.

Accordingly, since the DSs are provided at points likely to be chosen asplayback start points, during random access, if it is possible to decodeone of the graphics streams, it is also necessarily possible to decodethe other graphics stream. Thus, ensuring stereoscopic view of graphicsduring random access is possible.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages, and features of the invention willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings, which illustrate specificembodiments of the present invention.

In the drawings:

FIG. 1 shows a structure of a home theater system;

FIG. 2 shows an internal structure of a BD-ROM;

FIG. 3 shows an application format of the BD-ROM;

FIG. 4 shows a process through which source packets constituting a leftview stream and a right view stream are written into an AV data area;

FIG. 5 shows correspondences between physical units of a BD-ROM andsource packets constituting one file extent;

FIG. 6 shows a plurality of numerical ranges into which the packet IDsof TS packets may fall, and PES stream types of TS packets of the packetIDs in the numerical value ranges, in correspondence with each other;

FIG. 7 shows an interleaving arrangement;

FIG. 8 shows an example of internal structures of a BaseView stream andan EnhancedView stream;

FIG. 9 shows what type of video is provided for playback when switchingbetween transparency/opacity of goggles at the timing shown in FIG. 8;

FIG. 10 shows a stereoscopic effect due to an afterimage effect in theeye;

FIG. 11 shows a structure of a PG stream;

FIG. 12 shows a logical structure composed of various types offunctional segments;

FIG. 13 shows a data structure of ODS and PDS;

FIG. 14 shows a data structure of WDS and PCS;

FIG. 15 shows DS types, composition_numbers, and forced_on flags for DSscorresponding to the left view PG stream and the right view PG stream;

FIG. 16 shows a notation example of WDS and PCS that belong to aleft-view DS;

FIG. 17 shows a notation example of WDS and PCS that belong to aright-view DS;

FIG. 18 shows object ids of DSs corresponding to a left-view PG streamand a right-view PG stream;

FIG. 19 indicates a playback time axis of an AV clip to which a DSn hasbeen allocated;

FIG. 20 shows an example of a clip information file;

FIG. 21 shows an internal structure of an entry map table;

FIG. 22 shows a registration of an entry point on an entry map;

FIG. 23 shows how entry maps corresponding to each of the left view andthe right view are set;

FIG. 24 shows a data structure of playlist information;

FIG. 25 shows a data structure of a subpath information table;

FIG. 26 shows what type of playback sections are defined for the leftview and right view;

FIG. 27 shows an internal structure of a video stream number table;

FIG. 28 shows an internal structure of a PG stream information table inan STN_table;

FIG. 29 shows an internal structure of extension data in playlistinformation;

FIG. 30 shows an internal structure of a video stream number table;

FIG. 31 shows an internal structure of a playback apparatus;

FIG. 32 shows an internal structure of a playback unit 1;

FIG. 33 is a flowchart showing a processing procedure of playlistplayback processing;

FIG. 34 is a flowchart showing a playback procedure based on anSTN_table_extension;

FIG. 35 is a flowchart showing a procedure for setting PSR2s when anapparatus state changes and when a stream change is required;

FIG. 36 is a flowchart indicating a PSR2 setting procedure;

FIG. 37 shows an internal structure of a playback unit 1 in variation1-1;

FIG. 38 shows an IG stream structure;

FIG. 39 shows a logical structure composed of a variety of types offunctional segment;

FIG. 40 shows correspondences between ICS and Interactive_composition;

FIG. 41 shows an internal structure of ICS;

FIG. 42 shows an internal structure of page information for an arbitraryvalue (a number of pages x) among a plurality of pages;

FIG. 43 shows a notation example of ICS belonging to a left view DS;

FIG. 44 shows a notation example of ICS belonging to a right view DS;

FIG. 45 shows an internal structure of the playback unit 1 in embodiment2;

FIG. 46 pertains to a restriction of multiplexing order of a left viewPG stream and a right view PG stream;

FIG. 47 shows a variation of a PCS data structure, and

FIG. 48 shows a DS that includes a 3D-PCS.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Embodiments of the present invention are described below with referenceto the drawings.

Embodiment 1

Overall Structure

FIG. 1 shows a mode pertaining to a usage act of a recording medium anda playback apparatus. As shown in FIG. 1A, a BD-ROM 100 that is anexample of the recording medium, and a playback apparatus 200 constitutea home theater system together with a television 300, a pair of liquidcrystal shutter goggles 400, and a remote control 500, and are providedfor use by a user.

The BD-ROM 100 supplies, for example, a movie to the home theatersystem.

The playback apparatus 200 is connected to the television 300, and playsback the BD-ROM 100.

The television 300 provides an interactive operation environment to theuser by displaying the video playback of a movie, displaying a menu,etc.

The liquid crystal shutter goggles 400 are composed of a pair of liquidcrystal shutters and a control unit, and realize a stereoscopic viewwith use of the parallax between the user's eyes. The liquid crystalshutters of the liquid crystal shutter goggles 400 use liquid crystallenses having a property that the light transmittance of the lenschanges depending on the applied voltage. The control unit of the liquidcrystal shutter goggles 400 receives, from the playback apparatus, asynchronization signal indicating the switching between the output of aright view image and a left view image, and switches between a firststate and a second state according to this synchronization signal.

FIG. 1B shows the first state. In the first state, the applied voltageis so adjusted to make the liquid crystal lens corresponding to theright view becomes non-transparent and the liquid crystal lenscorresponding to the left view becomes transparent. In this state, animage for the left view is supplied for viewing.

FIG. 1C shows the second state. In the second state, the applied voltageis so adjusted to make the liquid crystal lens corresponding to theright view become transparent and the liquid crystal lens correspondingto the left view become non-transparent. In this state, the image for aright view is supplied for viewing.

In general, the right view and the left view of the human eyes perceiveslightly different images due to the positional difference therebetween.This positional difference makes it possible for humans to perceiveimages visible to the eye as a stereoscopic image. In view of this, bysynchronizing the timing at which the liquid crystal shutter goggles 400switches between the first state and the second state as described abovewith the timing of switching the output between the right view image andthe left view image, the user experiences an illusion that a flatsurface is stereoscopic.

Next, the following describes a time interval for displaying the rightview image and the left view image.

Specifically, in a monoscopic display image, there is a difference inviewing between a right view and a left view corresponding to thebinocular parallax, and a display that appears stereoscopic is achievedby performing display by switching between the right view and the leftview in a short time period.

The duration of each time interval should be short enough to create anoptical illusion that a three-dimensional (i.e., stereoscopic) image isdisplayed.

Returning to FIG. 1A, the remote control 500 is a device for receivinguser operations on a hierarchical GUI. In order to be able to receivesuch user operations, the remote control 500 is provided with a menu keyfor calling up a menu constituting the GUI, arrow keys for moving thefocus from one GUI component to another GUI component of the menu, anenter key for activating a currently focused GUI element of the menu,and a return key for returning to a higher hierarchical page of themenu, and numeric keys.

This concludes the description of the home theater system.

Recording Medium

Next, the following describes the recording medium targeted for playbackby the playback apparatus 200. The recording medium that is played backby the playback apparatus 200 is the BD-ROM 100. FIG. 2 shows theinternal structure of the BD-ROM 100.

The first row depicts the BD-ROM, which is a multi-layered optical disk.The second row depicts a spiral track on each recording layer in amanner of being extended in a horizontal direction. The spiral track ishandled as one contiguous recording area. The recording area is composedof a lead-in area located at the innermost position, a lead-out locatedat the outermost position, and the recording areas on the first, second,and third recording layers located between the lead-in and lead-outareas.

The third row depicts the file system area of the BD-ROM. The filesystem area is composed of a “volume management area” and a “logicaladdress space”.

The “volume management area” is an area having file system managementinformation stored therein. The file system management information isused to handle the respective recording areas on the first, second, andthird recording layers as one contiguous file system space.

The “logical address space” is an address space in which sectors areaddressable with consecutive logical block numbers (LBNs). That is, therespective recording areas on the first, second, and third recordinglayers depicted on the second row constitute one contiguous logicaladdress space.

The fourth row depicts the allocation of the logical address space inthe file system management area. The file system management area has anon-AV data recording area at the innermost position and also has an AVdata recording area immediately following the non-AV data recordingarea.

The fifth row depicts extents recorded in the non-AV data recording areaand the AV data recording area. In the AV data recording area, extents(EXT, EXT, EXT . . . in the figure) constituting an AV file arerecorded. In the non-AV data recording area, extents (EXT, EXT, EXT . .. in the figure) constituting a file other than an AV file are recorded.

FIG. 3 depicts an application format of the BD-ROM.

The “BDMV directory” is a directory for storing such data as AV contentsand management information handled on the BD-ROM. The BDMV directory hasfive sub-directories called “JAR directory”, “BDJO directory”, “PLAYLISTdirectory”, “CLIPINF directory”, and “STREAM directory”.

The BDMV directory contains two types of files: “index.bdmv” and“MovieObject.bdmv”.

The “index.bdmv” file contains management information relating to theentire BD-ROM. The index.bdmv file is read first after the disk isinserted to the playback apparatus, which enables the playback apparatusto uniquely identify the disk. In addition, the index.bdmv fileindicates, for each of a plurality of titles available for playback onthe BD-ROM, a title number and a BD-J object or movie object definingthe title.

The “MovieObject.bdmv” contains one or more movie objects. Each movieobject is a management object defining the control procedure to beexecuted by the playback apparatus in an operation mode (HDMV mode) inwhich a command interpreter is a control entity. The movie objectincludes one or more commands and a mask flag indicating whether or notto mask a menu call or a title call made by a user.

The “JAR directory” contains a JAR file corresponding to an archivefile. The archive file is created by combining one or more class filesand one or more data files into a single file. The one or more classfiles and one or more data files are combined into one file with the useof, for example, an archiver (not depicted).

The description below is directed to a Java (registered trademark)archive file as an example of the archive file.

For example, the Java (registered trademark) archive file defines thecontrol procedure to be executed by the playback apparatus in anoperation mode (BD-J mode). The control entity in the BD-J mode is aJava virtual machine, which is a bytecode interpreter, provided withinthe playback apparatus. A file containing a JAR file is identified by a5-digit number “zzzzz” and the extension “jar”.

The “BDJO directory” is a directory in which a file containing amanagement object (BDJ object) is placed. The BDJ object defines thecontrol procedure to be executed by the playback apparatus in theoperation mode (BD-J mode). The control entity in the BD-J mode is theJava virtual machine, which is a bytecode interpreter provided withinthe playback apparatus. A file containing a BDJ object is identified bya 5-digit number “zzzzz” and the extension “bdjo”.

The “PLAYLIST directory” is a directory in which a file containingPlayList information is placed. The PlayList information includesmain-path information specifying a playback section of a BaseView videostream and sub-path information specifying a playback section of anEnhancedView video stream. A file containing PlayList information isidentified by a five-digit number “yyyyy” and the extension “mpls”. TheBaseView video stream is a video stream for presenting a monoscopic(two-dimensional) display for one of the left view and the right view.On the other hand, a video stream for presenting the right view or theleft view and is not a BaseView video stream is referred to as an“EnhancedView video stream”. Picture data constituting the EnhancedViewvideo stream is compressed based on the screen correlation with picturedata constituting a corresponding BaseView video stream.

One example of a video compression scheme employing the correlationbetween the right and left views is Multiview Video Coding (MVC), whichis the video compression standard provided by amending MPEG-4 AVC/H.264.The Joint Video Team (JVT) is a joint project between ISO/IEC MPEG andITU-T VCEG and completed the development of MVC, which is an amendmentto H.264/MPEG-4 AVC video compression standard, in July 2008. MVC isintended for collectively encoding a plurality of video images for aplurality of different views. In the encoding, the correlation betweenpast and future screens is used for encoding as well as the correlationbetween screens for different viewpoints to achieve a higher compressionefficiency as compared with compression performed separately for screensof different views.

Streams constituting a BaseView and an EnhancedView are not limited tovideo streams. Presentation graphics (PG) streams may also constituteBaseView and EnhancedView. In the following description, a “BaseViewstream” may refer to either of a BaseView video stream and a BaseView PGstream, and an “EnhancedView stream” refers to an EnhancedView videostream and an EnhancedView PG stream.

The “CLIPINF directory” is a directory in which a file containing clipinformation (clip information file) is placed. A clip information fileis identified by a five-digit identifier “xxxxx” and the extension“clpi” and includes entry maps. One of the entry maps is of a videostream for the left view and another is of a video stream for the rightview.

The extents constituting files contained in the directories mentionedabove are recorded in the non-AV data area.

The “STREAM directory” is a directory for storing an AV clip filecontaining a monoscopic video stream and an AV clip file containing astereoscopic video stream. A file containing a monoscopic video streamis identified by a five-digit identifier “xxxxx” and the extension“m2ts”. A file storing a stereoscopic video stream is identified by afive-digit identifier “xxxxx” and the extension “ilts”.

The extents constituting a file containing a BaseView stream and placedin the STREAM directory and the extent constituting a file containing anEnhancedView stream and placed in the STREAM directory are recorded inthe AV data recording area.

FIG. 4 shows the processes through which the source packets of aBaseView stream and of an EnhancedView stream are recorded into the AVdata area. In the figure, the first row depicts TS packets constitutingthe BaseView stream or EnhancedView stream.

Each TS packet of 188 bytes that constitutes the BaseView stream or theEnhancedView stream is attached with a 4 byte TS_extra_header (hatchedbox in the figure) as depicted in the second row of the figure, whichresults in a sequence of source packets of 192 bytes. TheTS_extra_header includes an Arrival_Time_Stamp indicating the time atwhich the TS packet is to be input to the decoder.

The source packets of the BaseView stream and EnhancedView streamconstitute one or more “ATC sequences”. An “ATC sequence” refers to anarrangement of source packets constituting the time axis for ATSswithout any discontinuity (i.e., no arrival time-base discontinuity) inthe values of Arrival_Time_Clock fields referred to by theArrival_Time_Stamp fields. In other words, an “ATC sequence” refers to asequence of source packets having a successive sequence ofArrival_Time_Clock fields referred to by the Arrival_Time_Stamp fields.As will be described below, an ATS is attached at the head of a TSpacket and indicates the time at which the TS packet is to betransferred to the decoder.

Such an ATC sequence constitutes an AV clip and recorded on therecording layers with the file name “xxxxx.m2ts”.

Similarly to any normal computer file, such an AV clip is divided intoone or more file extents and recorded in the area on the recordinglayers. The third row of FIG. 4 depicts an AV clip and the fourth rowschematically depicts how the AV clip is recorded. The file extentsdepicted on the fourth row as constituting a file has a data lengthequal to or greater than a predetermined size (the size is calledS_EXT).

FIG. 5 depicts the relation between the physical units of the BD-ROM andthe source packets constituting one file extent. As depicted on thesecond row in the figure, a plurality of sectors are formed in the AVfile recording area of the BD-ROM. The source packets of the file extentis divided into groups of a plurality of source packets and recordedinto contiguous sectors, as depicted in the first row. The plurality ofsource packets are collectively referred to as an “Aligned Unit” andwriting to the BD-ROM is carried out in Aligned Units.

As depicted on the third row of the figure, an error-correcting code isattached to every plurality of sectors to constitute an ECC block. Aplayback apparatus is ensured to acquire a complete set of sourcepackets as long as an access to the BD-ROM is made in Aligned Units.This concludes the description of the process of writing an AV clip tothe BD-ROM.

FIG. 6A is a table showing possible numerical ranges of packet IDs of TSpackets (PID) and the types of PES streams carried by TS packets havingthe respective packet IDs.

TS packets having the packet ID “0x0100” constitute a program map(Program_map), whereas TS packets having the packet ID “0x1001”constitute a program clock reference (PCR).

TS packets having the packet ID “0x1011” constitute a BaseView videostream, and TS packets having the packet ID “0x1012” constitutes anEnhancedView video stream.

TS packets having packet IDs ranging from “0x1100” to “0x111F”constitute an audio stream.

TS packets having packet IDs ranging from “0x1220” to “x123F” constitutea BaseView PG stream. TS packets having packet IDs ranging from “0x1240”to “0x125F” constitute an EnhancedView PG stream. Note that TS packetsconstituting a PG stream for a two-dimensional view and not a BaseViewPG stream are assigned with the packet IDs ranging from “0x1200” to“0x121F”.

The TS packets constituting video streams and TS packets constituting PGstreams are grouped together depending on whether the TS packetsconstitute the BaseView or the EnhancedView. FIG. 6B shows one example.

As shown in FIG. 6B, the group of source packets constituting theBaseView include: source packets (each depicted as “Video” in thefigure) having the PID “0x1011” and constituting the BaseView videostream; source packets (each depicted as “Audio”) having the PID“0x1100” and constituting an audio stream; and source packets (eachdepicted as “PG”) having the PIDs “0x1220”, “0x1221”, “0x1222”,“0x1223”, “0x1224”, “0x1225”, and “0x1226” and constituting a PG stream.

On the other hand, the group of source packets constituting theEnhancedView include: source packets (each depicted as “Video”) havingthe PID “0x1012” and constituting an EnhancedView video stream; sourcepackets (each depicted as “Audio”) having the PID “0x1101” constitutingan audio stream; and source packets (each depicted as “PG”) having thePIDs “0x1240”, “0x1241”, “0x1242”, “0x1243”, “0x1244”, and “0x1245” andconstituting a PG stream.

The source packets belonging to the BaseView group and the EnhancedViewgroup are interleaved. FIG. 7 shows one example of the interleavedarrangement of the source packets. In the example shown in the figure,extents constituting the BaseView and the EnhancedView are alternatelyrecorded in the order of “BaseView”, “EnhancedView”, “BaseView” and“EnhancedView” . . . .

In FIG. 7, the first row depicts an AV file, and the second row depictsthe extents EXT_L[i], EXT_L[i+i], EXT_R[i], and EXT_R [i+1] constitutingthe AV file. The third row depicts a sequence of source packetsbelonging to the extents, and the fourth row depicts a sequence ofsectors on a recording layer. Note that variables “i” and “i+1” in thebrackets indicate the playback order of the respective extents.According to this notation, the two extents attached with the variable“i”, namely EXT_L[i] and EXT_R[i], are to be played synchronously.Similarly, the two extents attached with the variable “i+1”, namelyEXT_L[i+1] and EXT_R[i+1], are to be played synchronously.

The extents EXT_L[i] and EXT_L[i+1] are constituted by the sourcepackets having the PID “0x1011”. Dashed arrows h1, h2, h3, and h4indicate the membership, i.e., which of the BaseView stream andEnhancedView stream each of the extents EXT_L[i] and EXT_L [i+1] belongsto. According to the membership indicated by the arrows h1 and h2, theextents EXT_L[i] and EXT_L[i+1] belong to the BaseView stream and theextents EXT_R[i] and EXT_R[i+1] belong to the EnhancedView stream asindicated by the arrows h3 and h4.

The size of the extent EXT_L[i] is referred to as SEXT_L[i], whereas thesize of the extent EXT_R[i] is referred to as SEXT_R[i].

The following describes how to determine the size SEXT_L and SEXT_R.Note that in a playback apparatus, extents are read alternately to tworead buffers, one for the right view and the other for the left view,before being supplied to a video decoder. In view of this, SEXT_L andSEXT_R need to be determined in view of the time taken before therespective read buffers for the right view and the left view becomefull. When Rmax1 denotes the transfer rate to the right-view readbuffer, the capacity of the right-view read buffer needs to bedetermined to satisfy the following relation:Right-View Read Buffer=Rmax1×“Time Taken to Fill Left-View Read Bufferduring operation involving a jump”

The term “jump” is synonymous with a disk seek. The contiguous area thatcan be secured for recording in the BD-ROM is limited. Therefore, aBaseView stream and an EnhancedView stream are not necessarily locatednext to each other on the BD-ROM, and rather are recorded in discreteareas.

The following now considers the “Time Taken to Fill Left-View ReadBuffer during operation involving a jump”. TS packets are transferred tothe left-view read buffer at the transfer rate Rud−Rmax2. This Rud−Rmax2indicates the difference between the output rate Rmax2 of the left-viewread buffer and the input rate Rud of the left-view read buffer. Then,the time taken to fill the left-view read buffer is expressed asRB2/(Rud−Rmax2).

Regarding the reading of data to the left-view read buffer, it isnecessary to consider the jump time (Tjump) from a right-view videostream to a left-view video stream and the jump time (Tjump) from theleft view video stream to the right view video stream. Thus, the timeexpressed as (2×Tjump+RB2/(Rud−Rmax2)) is necessary to fill theleft-view read buffer.

Let Rmax1 denote the transfer rate of the right-view read buffer. Then,all the source packets in the right-view read buffer need to be outputat the transfer rate Rmax1 within the storage time of the left-view readbuffer. Therefore, the size RB1 of the right-view read buffer is givenby the following expression:RB1≧Rmax1×{2×Tjump+RB2/(Rud−Rmax2)}

In a similar manner, the size RB2 of the left-view read buffer is givenby the following expression:RB2≧Rmax2×{2×Tjump+RB1/(Rud−Rmax1)}

Specifically, the memory size of the right-view read buffer and of theleft-view read buffer is equal to 1.5 MB or less. According to thepresent embodiment, the extent size SEXT_R and the extent size SEXT_Lare determined to be equal or substantially equal to the right-view readbuffer and the left-view read buffer, respectively. This concludes thedescription of how the BaseView stream and EnhancedView stream arerecorded.

The following describes an internal structure of a BaseView stream andan EnhancedView stream.

FIG. 8 depicts an internal structure of a stereoscopic pair of aBaseView stream and an EnhancedView stream for stereoscopic viewing.

Each of the BaseView stream and the EnhancedView stream contains picturedata, for example. There many types of picture data, includingI-picture, P-picture, and B-picture.

An I-picture is picture data corresponding to one screen.

A P-picture is picture data representing the difference from theI-picture to which the P-picture refers.

A B-picture is picture data generated with reference to both anI-picture and a P-picture.

The second row of this figure depicts an internal structure of theBaseView stream. This stream contains pieces of picture data I1, P2,Br3, Br4, P5, Br6, Br7, and P9.

These pieces of picture data are decoded with the timing indicated bythe DTSs (decoding time stamp: indicating the time at which decoding ofthat piece of picture data by a decoder is to be started) attached tothe respective pieces of picture data. The first row depicts left-viewpictures and the decoded pictures I1, P2, Br3, Br4, P5, Br6, Br7, andP9. The left-view images are presented by executing playback of thepictures in the order of I1, Br3, Br4, P2, Br6, Br7, and P5 according tothe values of PTSs attached to the respective pictures.

The fourth row depicts an internal structure of the EnhancedView stream.This secondary video stream contains pictures P1, P2, B3, B4, P5, B6,B7, and P8. Those pieces of picture data are decoded with the timingindicated by the DTSs attached to the respective pieces of picture data.The third row depicts the right-view images. The right-view images arepresented by executing playback of the decoded pictures P1, P2, B3, B4,P5, B6, B7, and P8 in the order of P1, B3, B4, P2, B6, B7, and P5according to the values of PTSs (Presentation Time Stamp: informationindicating the presentation times of video and audio of that picture)attached to the respective pictures.

The fifth row depicts how the states of the LC shutter goggles 400 isswitched. As depicted on the fifth row, one of the LC shutterscorresponding to the right view is closed during the time that left-viewimages are presented, whereas the other of the LC shutters correspondingto the left view is closed during the time that right-view images arepresented.

Also, the EnhancedView streams are compressed by the intra-pictureprediction using the timewise redundancy between pictures as well theredundancy between pictures corresponding to different views. That is tosay, the pictures of the EnhancedView stream are compressed withreference to pictures of the BaseView stream having the samepresentation time.

For example, the first P-picture in the EnhancedView stream refers to anI-picture in the BaseView stream, and a B-picture in the EnhancedViewstream refers to a Br-picture in the BaseView stream. The secondP-picture in the EnhancedView stream refers to a P-picture in theBaseView stream.

FIG. 9 illustrates playback images presented to the user by switchingthe transparent/opaque states of the goggles with the timings depictedin FIG. 8. Suppose that the screen display period is 1/24 sec and thetransparent/opaque states of the individual goggles corresponding to theright view and left view are switched at every 1/48 sec. Consequently,pictures corresponding to the right view and the left view arealternately presented one by one. FIG. 9 schematically depicts thatimages presented for the left view are slightly different from imagespresented for the right view, with respect to the angle and/or positionof the face of a person appearing in the images (Note that thedifference in the face angle and position in FIGS. 9 and 10 are onlyschematic illustrations).

FIG. 10 illustrates a stereoscopic image created by an afterimage effectin the human eye. Switching the transparent/opaque states of the gogglesaccording to the timing shown in FIG. 8 enables causing recognition as astereoscopic image.

The following describes PG streams. FIG. 11 depicts a structure of a PGstream. In FIG. 11, the first row depicts a sequence of TS packetsconstituting an AV Clip and the second row depicts a sequence of PESpackets constituting the PG stream. The sequence of PES packets on thesecond row are acquired by extracting and concatenating payloads of TSpackets having a predetermined PID from among the TS packets on thefirst row.

The third row depicts a structure of the PG stream. The PG stream iscomposed of functional segments including PCS (Presentation compositionsegment), WDS (Window Definition Segment), PDS (Palette DefinitionSegment), ODS (Object Definition Segment), and END (END of Display SetSegment). Of these functional segments, a PCS is called a displaycomposition segment, and WDS, PDS, ODS, and END are called definitionsegments. PES packets and functional segments may be in one-to-onecorrespondence or in one-to-multiple correspondence. That is, onefunctional segment may be converted into one PES packet or fragmentedinto a plurality of PES packets and recorded on the BD-ROM.

FIG. 11B depicts PES packets acquired by converting functional segments.As depicted in FIG. 11B, each PES packet is composed of a “packetheader” and “payload”, which is the data entity of the functionalsegment. The packet header contains a DTS and a PTS of the functionalsegment. In the following description, the DTS and PTS contained in thePES packet header of a functional segment is referred to as the DTS andPTS of the functional segment.

The functional segments of the various types mentioned above constitutea logical structure as shown in FIG. 12. FIG. 12 depicts the logicalstructure constituted by the various types of functional segments. Inthe figure, the fourth row depicts the functional segments, the thirdrow depicts types of Display Sets, the second row depicts Display Sets,and the first row depicts Epochs. Each Display Set (hereinafter, simply“DS”) depicted on the second row is a set of functional segmentsconstituting one screen of graphics data, or of graphics data containedin the PG stream. Also, a broken line kz1 in the figure indicates themembership, i.e., which of the functional segments depicted on thefourth row belongs to which DS. It is shown that a series of functionalsegments PCS-WDS-PDS-ODS-END constitute one DS. A playback apparatus iscapable of configuring one screen of graphics data by reading from theBD-ROM a plurality of functional segments constituting the DS.

Each Epoch depicted on the first row refers to a period, on the playbacktime axis of the AV Clip, over which the memory management iscontinuous, and also refers to a group of data allocated to that period.The memory to be managed includes a graphics plane for storing onescreen of graphics data and an object buffer for storing decompressedgraphics data. The memory management is said to be continuous if noflash of the graphics plane and the object buffer occurs through theEpoch and erasing and rendering of graphics data occurs only during alimited rectangular area of the graphics plane. (*Please note that to“flash” means to clear the stored content of the entire Graphics Planeand the entire Object Buffer.) In other words the rectangular area isfixed in size and position over the Epoch. When deletion and renderingof graphics are performed within the fixed rectangular area,synchronization of graphics and video is ensured. In other words, theEpoch is a time unit on the playback time axis during which thesynchronization between video and graphics is ensured. To change thepredetermined area for which rendering/deletion is performed, it isnecessary to define a change point on the playback time axis and set anew Epoch from that point forward. At the boundary between such twoEpochs, the synchronization between video and graphics is not ensured.

Also, described with reference to the position of subtitles on thedisplay screen, an Epoch is said to be a period, on the playback timeaxis, during which subtitles are displayed only within a fixedrectangular area of the display screen.

This completes the description of Epochs. The following now describesDisplay Sets.

In FIG. 12, the broken lines hk1, 2 indicate to which type of DS in thethird row each DS in the second row belongs. Also, the series of DSsincluding Epoch Start, Acquisition Point, and Normal Case constitute anEpoch depicted on the first row. Note that “Epoch Start”, “AcquisitionPoint”, and “Normal Case” denote types of DSs. Though the AcquisitionPoint DS precedes the Normal Case DS in FIG. 12, this is merely anexample and the order may be reversed.

An “Epoch Start” is a Display Set that indicates the start of a newEpoch. For this reason, an Epoch Start contains a complete set offunctional segments necessary for constituting a display composition. AnEpoch Start is provided at such a position that is likely to be chosenas a playback start point and the beginning of a chapter in a movie isone example.

An “Acquisition Point” is a Display Set that is not the start of anEpoch but contains a complete set of functional segments necessary forconstituting a subsequent display composition. An Acquisition Point DSensures that graphics are duly displayed as long as playback is startedfrom the Acquisition Point DS. In other words, the Acquisition Point DSenables screen composition from a midpoint in the Epoch. EachAcquisition Point DS is provided at a position that may be chosen as astarting point, such as a position that is specifiable using a timesearch. “Time search” refers to an operation by which a user inputs anumber of minutes/seconds to locate a corresponding playback point. Thetime is specified by a user input in units of, say, 10 seconds (or 10minutes), so that playback points positioned at intervals of 10 seconds(or 10 minutes) will be specifiable using the time search. TheAcquisition Point DSs are provided in positions that are specifiableusing the time search, so that playback of the PG stream is dulyexecuted in response to a time search.

A “Normal Case” DS contains only difference data relative to theprevious Display Set. For example, if a DSv graphic has the same contentas an immediately preceding DSu but has a different display compositionfrom the DSu, the DSv will be a Normal Case DS containing only a PCS andan END. With this arrangement, there is no need to provide ODSs thathave already been provided. As a result, the amount of data stored onthe BD-ROM can be reduced. On the other hand, it is not possible todisplay graphics using a Normal Case DS alone since the DS containsdifference data only.

The following describes the Definition Segments (ODS, WDS, and PDS).

An “Object_Definition_Segment” is a functional segment that defines agraphics object. The following describes graphics objects.

AV clips recorded on BD-ROMs feature an image quality equivalent to thatof high-definition television pictures.

The resolution for displaying graphics objects is set to an equivalentlyhigh 1920×1080. This high resolution permits reproduction of movietheater style subtitles (e.g. crisp handwriting style subtitles) andanimated characters on BD-ROMs, for example. A graphics object iscomposed of a plurality of pieces of run-length data. Run-length dataexpresses a pixel string using a pixel code that indicates a pixel valueand a continuous length of the pixel value. The pixel code is 8 bits inlength, and takes a value of 1 to 255. The run-length data is able toselect 256 colors of a possible 16,777,216 in accordance with the pixelcode, and set the pixel color. Note that when a graphics object is forpresenting subtitles, it is necessary to dispose a text string on atransparent background.

The ODS defines PGs according to a data structure shown in FIG. 13A. Asshown in FIG. 13A, the ODS is composed of the following fields:“segment_type” indicating that the segment is of the ODS type;“segment_length” indicating the data length of the ODS; “object_id”uniquely identifying the graphics object corresponding to PGs in theEpoch; “object_version_number” indicating a version of the ODS in theEpoch; “last_in_sequence_flag”; and “object_data_fragment” containing aconsecutive sequence of bytes corresponding to part or all of the PGs.

The Epoch of the graphics stream includes a plurality of ODSs having thesame ID. The plurality of ODSs having the same ID also have the samewidth and height, and are assigned with a common area in the ObjectBuffer. After one of the ODS having the same ID is read in the commonarea, the read ODS is overwritten by a subsequent ODS having the sameID. In this way, according to the playback progression of the videostream, the ODSs in the object buffer being written over by thesubsequently read ODSs that have the same identifier, enable thepictures in the ODS to achieve a temporal transition. The restrictionthat graphics objects having the same ID also have the same vertical andhorizontal size is only applied within the same Epoch. Graphics objectsbelonging to a certain Epoch, and graphics objects belonging to the nextEpoch, may have different vertical and horizontal sizes.

Explanations about the last_insequence_flag and object_data_fragment aregiven next. In some cases, it is not possible to store the decompressedgraphics that constitute the PG in one ODS due to a payload constraintof the PES packet. In such cases, the graphics are split into a seriesof consecutive fragments, and one fragment is set to theobject_data_fragment. When one Graphics Object is stored as more thanone fragment, every fragment except a last fragment has the same size.The last fragment is less than or equal to the size of previousfragments. The ODS carrying the fragments appear in the same sequentialorder in the DS, with an end of the sequence indicated by the ODS havingthe last_insequence_flag. Although the above described syntax of the ODSis based on a premise that the fragments are stacked in from thepreceding PES, the fragments may be stacked so that each PES contains ablank part.

A “Palette Definition Segment (PDS)” is a functional segment storingpalette data. The palette data indicates combinations of pixel codes of1 to 255 and pixel values. Each pixel value is composed of a reddifference component (Cr value), a blue difference component (Cb value),a luminance component (Y value), and a transparency (T value). Eachpixel code in the run-length data is converted into a pixel valueindicated by the palette to generate a color. A data structure of PDS isshown in FIG. 14B. As shown in FIG. 14B, the PDS includes the followingfields: “segment_type” indicating that the segment is of the PDS type;“segment_length” indicating the data length of the PDS; “palette_id” foruniquely identifying the palette included in the PDS,“palette_version_number” indicating a version of the PDS within theEpoch; and “palette_entry” carrying information for each entry. The“palette_entry” field indicates a red difference component (Cr value), ablue difference color component (Cb value), a luminance component (Yvalue), and a transparency (T value) for each entry.

The following describes a WDS.

A “window_definition_segment” is a functional segment that defines arectangular area on the Graphics Plane. As described above, the memorymanagement is continuous over an Epoch on condition that clearing andrendering takes place only within a fixed rectangular area on theGraphics Plane. The rectangular area on the Graphics Plane is called a“Window”, and is defined by a WDS. FIG. 15A depicts a data structure ofa WDS. As shown in the figure, a WDS includes the following fields:“window_id” uniquely identifying the Window on the Graphics Plane;“window_horizontal_position” indicating the horizontal position of a topleft pixel on the Graphics Plane; “window_vertical_position” indicatinga vertical position of a top left pixel on the Graphics Plane;“window_width” indicating a width of the Window on the Graphics Plane;and “window_height” indicating a height of the Window on the GraphicsPlane.

The following describes the values that window_horizontal_positionfield, the window vertical_position_field, and the window_width fieldmay take. These fields relate to the coordinate system defining theinternal area of the graphics plane. This Graphics Plane has atwo-dimensional area having a size defined by the values of video_heightand video_width.

The window_horizontal_position field specifies a horizontal position ofthe top left pixel of the Window on the Graphics Plane, and thus takesvalue in a range of 1 to the video_width. The window_vertical_positionfield specifies the vertical position of the top left pixel of theWindow on the Graphics Plane, and thus takes a value in a range of 1 tothe video_height.

The window_width field specifies the width of the Window on the GraphicsPlane, and thus takes a value in a range of 1 to(video_width)−(window_horizontal_position). The window_height fieldspecifies the height of the Window on the Graphics Plane, and thus takesa value in a range of 1 to (video_height)−(window_vertical_position).

The position and size of a Window can be defined for each Epoch usingthe window_horizontal_position, window_vertical_position, window_width,and window_height fields in the WDS. Thus, adjustments may be made atthe time of authoring, such that the Window does not interfere withelements of the picture over the period of an Epoch. For instance, theWindow may be made to appear in a desired margin to ensure subtitlesdisplayed using graphics are clearly viewed. Since a WDS may be definedfor each Epoch, the graphics are displayed at a suitable position on thedisplay screen, as elements of the picture change with time. Thismaintains that graphics are displayed at suitable position for ensuringclear view of the graphics. As a result, the quality of the movieproduct can be raised to the level of a cinema film, in which subtitlesare integrated into the picture.

The following describes an “END of Display Set Segment”. The END ofDisplay Set Segment is a functional segment indicating that transmissionof a Display Set is complete, and is positioned immediately after thelast ODS in the Display Set. The END of Display Set Segment includes thefollowing fields: “segment_type” indicating that the segment is of theEND of Display Set type; and “segment_length” indicating the data lengthof the END of Display Set Segment. The fields of the END of Display SetSegment do not specifically require description and thus are notdepicted in the figures.

This concludes the description of ODS, PDS, WDS and END.

The following now describes a PCS. The PCS is a functional segment forconstituting an interactive screen. FIG. 14B depicts a data structure ofthe PCS. As shown in FIG. 14B, the PCS includes the following fields:“segment_type”; “segment_length”; “composition_number”;“composition_state”; “palette_update flag”; “palette_id”; and“composition_object (1) to (m)”.

The “composition_number” field identifies a graphics update in theDisplay Set, using a number from 0 to 15. Specifically, thecomposition_number field is incremented by one for each graphics updatefrom the beginning of the Epoch to the PCS containing thecomposition_number field.

The “composition_state” field indicates whether the Display Set startingfrom the PCS is a Normal Case, an Acquisition Point, or an Epoch Start.

The “palette_update_flag” field indicates whether the PCS describes aPalette-only Display Update. The Palette-only Display Update is anupdate that is limited to replacing the palette with a new palette. Toindicate a Palette-only Display Update, the palette_update_flag is setto 1.

The palette_id indicates a palette to be used for a Palette-only DisplayUpdate.

The “composition_object” fields (1) to (m) each contain controlinformation for realizing a screen composition using the Display Set towhich the PCS belongs. In FIG. 14B, a composition_object(i) is expanded,as indicated by broken lines wd1, to illustrate an internal structure ofthe composition_object fields. As illustrated, the composition object(i)includes the following fields: “object_id_ref”; “window_id ref”;“forced_on flag”; “object_cropped_flag”; “object_horizontal_position”;“object_vertical_position”; and “cropping_rectangle_information (1) (2). . . (n)”.

The “object_id_ref” field contains a reference value that is a PGidentifier (object_id). This reference value identifies the PGs to beused to present the display composition corresponding to thecomposition_object(i).

The window_id ref field contains a reference value that is awindow_identifier (window_id). This reference value specifies the Windowin which the PGs is to be displayed to present the display compositioncorresponding to the composition_object(i).

The “forced_on_flag” is 1 when indicating a graphics object to beforcibly displayed even when the graphics object is set to OFF in thesetting of the playback apparatus.

The “object_cropped_flag” field indicates whether or not PGs cropped inthe object buffer are to be displayed. When the object_cropped_flagfield is set to 1, the cropped PGs in the object buffer are displayed.When the object_cropped_flag field is set to 0, the cropped PGs in theobject buffer are not displayed.

The object_horizontal_position field specifies a horizontal position ofthe top left pixel of the PGs on the graphics plane.

The object vertical_position_field specifies a vertical position of thetop left pixel of the PGs on the graphics plane.

The “cropping_rectangle information(1), (2) . . . (n)” fields are validwhen the “object_cropped_flag” field is set to 1. The cropping_rectangleinformation(i) is expanded, as indicated by broken lines wd2, toillustrate an internal structure of the composition rectangleinformation. As indicated by the broken lines, the cropping_rectangleinformation(i) includes the following fields:“object_cropping_horizontal_position”;“object_cropping_vertical_position”; “object_cropping_width”; and“object_cropping_height”. The object_cropping_horizontal_position fieldspecifies a horizontal position of the top left corner pixel of acropping rectangle for PGs on the graphics plane. The cropping rectangleis used to crop a portion of the PGs and corresponds to a “Region” inthe ESTI EN 300 743 standard. The “object_cropping_vertical_position”specifies a vertical position of a top left pixel of the graphics objecton the graphics plane.

The “object_cropping_width” field specifies a width of the croppingrectangle in the graphics object.

The “object_cropping_height” field specifies a width of the croppingrectangle in the graphics object.

This concludes the description of the data structure of PCS.

Restriction when Using a 2-Decoder Model

The following describes how the fields of the above-described functionalsegments are set for each of the left-view PG stream and the right-viewPG stream, respectively.

FIG. 15 shows a correspondence between a type of DS, acomposition_number, and a forced_on_flag included in a left view PGstream, and a type of DS, a composition_number, and a forced_on flagthat are included in a right view PG stream. As shown in FIG. 15, the DSthat is an Epoch Start and the DS that is an Acquisition Point alwaysexist as a pair of left view and right view. The reason for this is thatthe Epoch Start and the Acquisition Point are a unit that can completedrawing of one or more graphics objects as a single unit, and 3Dplayback cannot be performed unless the Epoch Start and the AcquisitionPoint are a pair. In other words, when the DS for the left view is anEpoch Start, there is always a paired DS for the right view, and thatcomposition_state also must be an Epoch Start.

Similarly, when the left view DS is an Acquisition Point, a right viewDS always exists to be paired with the left view DS, and thatcomposition_state also must be an Acquisition Point.

When the left view DS is Normal Case, it is also preferable for thecorresponding right view DS to be Normal Case.

In this way, by making DSs that correspond share the same type, astereoscopic view can be ensured, for example when performing randomaccess. In other words, when the left view DS is for example, EpochStart, since the DS of the corresponding right view is also Epoch Start,arranging the left view DS and the right view DS in positions thatreveal that these display sets are skipped enables avoiding a situationin which only one of the left view and the right view is decoded and theother one cannot be decoded.

Also, as shown in FIG. 15, the composition_number of a left view DS mustbe equal to the composition_number of a right view DS that is to beplayed back as a pair with the left view DS. Making thecomposition_numbers equal for correlating the DSs for the left view andthe right view in this way enables the graphics decoder to easilydetermine which of the left view DS and the right view DS to output atthe time of playback.

Furthermore, it is necessary to set the forced_on flag in the DS of theleft view PG stream to the same value as the forced_on flag of the DScorresponding to the right view PG stream. The reason is that astereoscopic view of the graphics object is impossible when only theright view of the graphics object (or only the left view of the graphicsobject) is output during playback, and the corresponding left view ofthe graphics object (or right view of the graphics object) is notoutput.

The fact that presentation composition segments are stored in PESpackets has already been described. Of course, the PTS of the PES packetthat stores the presentation composition segment included in the leftview DS must be identical in value with the PTS of the PES packet thatstores the presentation composition segment included in the right viewDS.

Also, in the palette definition segment of the left view DS and thepalette definition segment of the corresponding right view DS, identicalcontent is set for correspondences between code values, luminosity andcolor difference, and the palette_ids are also identical for both.

Next, the following describes how the fields of WDS and the other fieldsdescribed above in the PCS are filled in. FIGS. 16 and 17 show examplesof writing of WDS and PCS belonging to a Display Set. Here, although anexample is described of a case in which the graphics are animatedcharacters, the graphics may be subtitles composed of decorative script,etc.

FIG. 16 shows an example of writing WDS and PCS belonging to the leftview DS. In FIG. 16A, the window_horizontal_position (left) field andthe window_vertical_position field value in the WDS specify the top leftcoordinates of the Window on the graphics plane. The window_width fieldand the window_height field in the WDS specify the height and width ofthe display screen of the window. Here, the (left) of thewindow_horizontal_position (left) means that thiswindow_horizontal_position has a different value from the correspondingwindow_horizontal_position (right).

In FIG. 16A the object_cropping_horizontal_position (left) field and theobject_cropping_vertical_position field that are included in the cropinformation specify a basepoint of a cropping rectangle in a coordinatesystem whose origin is the top left of the graphics object in the objectbuffer. The cropping rectangle (enclosed by a thick line) is thendefined using the values of the object_cropping_width field, theobject_cropping_height field, and the basepoint. The cropped graphicsobject is then disposed in a region cp1 (enclosed by a rectangle) sothat a top left corner of the cropped graphics object lies at a pixelspecified by an object_horizontal_position (left) field value and anobject vertical_position_field value in the coordinate system of thegraphics plane. This causes the graphics to be written into the windowof the graphics plane. Consequently, the graphics are overlaid with themoving picture.

FIG. 17 is a notation example of WDS and PCS belonging to a right viewDS. In FIG. 17A, the window_horizontal_position (right) andwindow_vertical_position of WDS indicate an top left coordinate of thewindow in the graphics plane, and the window_width and the window_heightindicate the width and height of the display screen of the window. Here,the “right” of the window_horizontal_position (right) means that thiswindow_horizontal_position is a different value from thewindow_horizontal_position (left) of the corresponding left view.

The object_cropping_horizontal_position (right) and theobject_cropping_vertical_position in the crop information of FIG. 17Aindicate a basepoint of a cropping range in the coordinate system thattakes the top left coordinate of the graphics in the object buffer as abase. The range indicated by the object_cropping_width, theobject_cropping_height, and this basepoint (indicated by a bold lineportion in the drawing) is the cropping range. The cropped graphics arearranged in the coordinate system of the graphics plane in therectangular range cp2 that has the object_horizontal_position (Right)and the object_vertical_position as the basepoint (top left). Thiscauses the graphics to be written into the window of the graphics plane.Consequently, the graphics are overlaid with the moving picture.

As shown above, regarding WDS, the window_width and window_height areset with the same values in both the left view and the right view. Thereason for this is that, when the horizontal widths and vertical widthsof the display screens of the windows are different between the leftview and the right view, there are cases in which the graphics fit inthe display screen of the window in one view, and do not fit in thedisplay screen in the other view, and synchronization between the videoand the graphics are not safeguarded in such a case. Also, regarding thetop left coordinate of the window in the graphics plane, the value ofthe window_vertical_position is the same for both the left view and theright view, but the value of the window_horizontal_position is differentbetween the left view and the right view. This is normally caused by thedifference in the position of a visible object in a horizontal directionbetween the left eye and the right eye. Depending on the method ofgenerating graphics, the values of the window_vertical_position may bedifferent between the left view and right view, and the values of thewindow_horizontal_position may be the same for both the left view andthe right view.

Also, the object_cropping_width and the object_cropping_height may beset with the same values for both the left view and the right view. Thereason for this is that when the cropping range is different, adifference results in the graphics arranged in the graphics planebetween the left view and the right view. Also, regarding the basepointof the cropping range, although the object_cropping_vertical_position isset with the same value in both the left view and the right view, theobject_cropping_horizontal_position is set with different values betweenthe left view and the right view. The values of theobject_cropping_vertical_position may be different between the left viewand the right view, and the value of theobject_cropping_horizontal_position may be the same between the leftview and the right view.

Also, regarding the basepoint of the rectangular range, although theobject_vertical_position is set with the same value between the leftview and the right view, the object_horizontal_position is set with adifferent value between the left view and the right view. As describedabove, this is normally caused by the difference in the position of avisible object in a horizontal direction between the left eye and theright eye. Depending on the method of generating graphics, the values ofthe object_vertical_position may be different between the left view andright view, and the values of the object_horizontal position may be thesame for both the left view and the right view.

Furthermore, as shown in FIGS. 16 and 17, since the graphics of the leftview and the graphics of the right view are graphics viewed fromdifferent angles, object_data_fragments of both ODSs are naturallydifferent. As shown in FIG. 18, the object_id is set with the samevalue. The reason for this is that, when using a 2-decoder structure inthe playback apparatus, it is possible to use a structure in which onegraphics controller is shared between the two decoders, and in thatcase, unless the object ids of the graphics objects corresponding to theleft view and the right view are set with the same value, the graphicscontroller cannot distinguish which graphics objects correspond to eachother.

Adding the above-described restriction enables realization of astereoscopic view in a 2-decoder structure by a left view PG stream anda right view PG stream.

The following describes the manner in which Display Sets including PCSsand ODSs are allocated to the playback time axis of the AV Clip. AnEpoch is a period on the playback time axis over which memory managementis continuous, and each Epoch constitutes one or more Display Sets. Atissue, therefore, is the manner in which the one or more Display Setsare allocated to the playback time axis. Note that the playback timeaxis is an assumed axis used to prescribe decoded timings and playbacktimings for each screen of picture data making up the video stream thatis multiplexed into the AV Clip. The decode timing and playback timingare expressed to an accuracy of 1/90000 seconds. The DTS and PTSassociated with the PCS and ODS in the Display Set specify timings forsynchronous control on the playback time axis. In other words, the DSsare allocated to the playback time axis by carrying out synchronouscontrol using the DTSs and PTSs associated with the PCSs and ODSs.

A Display Set DSn of the Display Sets belonging to the Epoch isallocated to the playback time axis using a PTS and DTS which are set asshown in FIG. 19. FIG. 19 shows the AV Clip playback time axis to whichthe DSn has been allocated. In FIG. 19, the start of the DSn period isspecified by a DTS value (DTS(DSn[PCS])) of a PCS belonging to the DSn,and the end of the period is specified by a PTS value (PTS(DSn[PCS])) ofthe same PCS. The timing for the first display is also specified by thePTS value (PTS(DSn[PCS])) of the PCS. If the timing of a desired picturein a video stream is matched with the PTS(DSn[PCS])), first display ofthe DSn will synchronize with the video stream.

The PTS(DSn[PCS]) is a value obtained by adding DTS(DSn{PCS}) a periodfor decoding ODS (DECODEDURATION) to the DTS(DSn[PCS]).

Decoding necessary for the first display of the DSn is performed in theDECODEDURATION. In FIG. 19, a period mc1 is a period during which anarbitrary ODS (ODSm) belonging to the DSn is decoded. The starting pointof the decode period mc1 is specified by DTS(ODSn[ODSm]), and the endpoint of the decode period mc1 is specified by PTS(ODSn[ODSm]).

The Epoch is therefore prescribed by allocating each of the ODSs in theEpoch to the playback time axis. This concludes the description of theallocation to the playback time axis.

Clip Information File

FIG. 20 shows an example of a clip information file. As shown in thefigure, each clip information file contains management information foran AV clip. The clip information files are provided in one to onecorrespondence with the AV clips, and are each composed of the clipinformation, a stream attribute table and an entry map table.

A lead line zh1 shows a close-up of an internal structure of a streamattribute table. As shown by the lead line, the stream attribute tablecontains, for each PID, a piece of attribute information of acorresponding stream included in the AV clip. The attribute informationincludes different information for each BaseView stream and EnhancedViewstream.

A lead line zh2 shows a close-up of an internal structure of theBaseView stream. As shown by the lead line, the stream attributeinformation of the BaseView stream composed of TS packets having thePID=0x1011 specify the codec, resolution, aspect ratio, and frame rate.

The following describes an internal structure of the entry map table.

The entry map table shows a relationship between a source packet numberof an arbitrary source packet and a PTS on an STC time axis, from amongSTC time axes specified with use of a packet ID.

The STC time axis is a time axis of a MPEG2-TS defined to indicate adecode time and a display time. An “STC sequence” refers to a series ofsource packets that are without any discontinuity in STC (System TimeClock), which is the system time-base for the AV streams.

FIG. 21A depicts an internal structure of the entry map table. A leadline eh1 shows a close-up of the internal structure of the entry maptable.

As shown by the lead line eh1, entry maps are provided one for eachpacketized elementary stream composed of a plurality of types of TSpackets. More specifically, one entry map is provided for a BaseViewstream composed of TS packets having the PID=0x1011, and one entry mapfor an EnhancedView stream composed of TS packets having the PID=0x1012.The entry maps indicates a plurality of “entry points” each of which isinformation made up of a pair of a PTS and an SPN. Each entry point has,in addition to the PTS and SPN, a display method flag (an“is_angle_change” flag) indicating whether decoding from that SPN ispossible. Also, a value that is incremented for each entry point from astart point of 0 is called an “entry point ID” (hereinafter referred toas “EP_ID”).

With use of this entry map, the playback apparatus is enabled to specifythe location of a source packet corresponding to an arbitrary point onthe time axis of the video stream. For example, when performing aspecial playback operation such as fast forward or rewind, the playbackapparatus is enabled to effectively execute the playback withoutanalyzing an AV clip, by selecting an I picture registered in the entrymap as the playback start point. Also, an entry map is provided for eachvideo stream multiplexed into the AV clip, and these entry maps aremanaged with use of PIDs.

The lead line eh2 shows a close-up of an internal structure of the entrymap of an elementary stream composed of source packets having thePID=0x1011. The entry map indicates entry points corresponding toEP_ID=0, EP_ID=1, EP_ID=2, and EP_ID=3. The entry point corresponding toEP_ID=0 is composed of an is_angle_change flag set to ON, an SPN=3, anda PTS=80000. The entry point corresponding to EP_ID=1 is composed of anis_angle_change flag set to OFF, an SPN=1500, and a PTS=270000.

The entry point corresponding to EP_ID=2 is composed of anis_angle_change flag set to OFF, an SPN=3200, and a PTS=360000. Theentry point corresponding to EP_ID=3 is composed of an is_angle_changeflag set to OFF, an SPN=4800, and a PTS=450000.

FIG. 21B shows source packets specified by the plurality of entry pointsin the entry map corresponding to the TS packets PID=0x1011 shown inFIG. 23A. The entry point corresponding to EP_ID=0 indicates SPN=3, andthis source packet number is associated with PTS=80000. The entry pointcorresponding to EP_ID=1 indicates SPN=1500, and this source packetnumber is associated with PTS=270000.

The entry point corresponding to EP_ID=2 indicates the source packethaving SPN=3200, and this source packet number is associated withPTS=360000. The entry point corresponding to EP_ID=3 indicates thesource packet having SPN=4800, and this source packet number isassociated with PTS=450000.

FIG. 22 depicts entry points registered in an entry map. The first rowdepicts a time axis specified by an STC sequence. The second row depictsan entry map in clip information. The third row depicts source packetsequences in the ATC sequence.

The arrows te1, te2, te3, and te4 schematically show a correspondencerelationship between the playback points t1, t11, t21, and t31 in theATC time axis and respective entry points. The arrows sh1, sh2, sh3, andsh4 schematically show a correspondence relationship between SPN=n1,n11, n21, and n31 and respective entry points.

Consider the entry map that specifies source packets having SPN=n1, fromamong the source packets in the ATC sequence. In the case of this entrymap, the PTS is set to indicate t1 in the STC sequence. This entry mapenables the playback apparatus to execute random access to SPN=n1 in theATC sequence, with use of the time point PTS=t1 in the STC sequence.Also, if the source packet SPN=n21 is specified from among the sourcepackets in ATC sequence, the PTS of the entry map is set to indicate t21in the STC sequence. This entry map enables the playback apparatus toexecute random access to SPN=n21 in the ATC sequence, with use of thetime point PTS=t21.

FIG. 23 shows how entry maps corresponding to each of the left view andthe right view are set. According to the correspondences in FIG. 23, asource packet number in the STC sequence is recorded as the sourcepacket number of each entry point, and a PTS in the STC sequence isrecorded as the PTS of each entry point. The entry maps indicate how thesource packets correspond to the time axis.

The arrows th1, th2, th3, and th4 schematically show the correspondencebetween playback time points t1 and t2 on the STC time axis and theentry points. The arrows sh1, sh2, sh3, and sh4 schematically show thecorrespondences between the SPN=n1, n11, n8, and n18 in the ATC sequenceand entry points.

The fifth row depicts extents of a left view and a right view recordedby interleaving, and is the same as the figures described above. Thethird row shows entry maps corresponding respectively to PID=0x1011 andPID=0x1012. The entry map corresponding to PID=0x1011 includes an entrypoint indicating n1 and an entry point corresponding to n8. These entrypoints indicate correspondences between n1 and n8, and t1 and t2 on theSTC time axis. The entry map corresponding to PID=0x1012 includes anentry point indicating n11 and an entry point indicating n18. Theseentry points show correspondences between n11 and n18, and t1 and t2 onthe STC time axis.

That is, left and right view extents that are to be played back at thesame time point on the time axis may be recorded in widely separatedpositions in the AV data recording area. Yet, with the use of entry mapscorresponding respectively to the extents, source packets at the startof the right view extent and the left view extent may be uniquelyaccessed based on the corresponding PTS.

This concludes the description of the clip information files.

PlayList Information

Next, the details of the PlayList information are described.

FIG. 24 depicts a data structure of PlayList information. In FIG. 24,the PlayList information includes playback attribute information,MainPath information, a SubPath information table, and extension data.

First, playback attribute information is described. The lead line mp3shows a close-up of an internal structure of playback attributeinformation. As shown with the lead line mp3, the playback attributeinformation includes “version number” indicating the version of thestandards on which the content is based, “playback type”, and“stereoscopic display flag”. The “version number” field may store avalue such as “2.00” indicating the version of the BD-ROM applicationformat. The “playback type” field causes a playback apparatus to execute“sequential” or “random/shuffle” playback. The sequential playback is toplayback PlayItems included in a PlayList in order form the beginning.

Next, MainPath information is described. The lead line mp1 shows a closeup of the internal structure of the MainPath information. As shown bythe arrow mp1, a MainPath is defined by a plurality of PlayIteminformation pieces #1 . . . #m. The PlayItem information defines onelogical playback section that makes up the MainPath. The structure ofthe PlayItem information is shown in close up by the lead line mp2. Asshown by the lead line, the PlayItem information is constituted fromsets of a “Clip_Information_file_name” that indicates a file name ofplayback section information of an AV clip belonging to the IN point andOut point of the playback section, a “Clip_codec_identifier” thatindicates an encoding method of an AV clip, an “is_multi_angle” thatindicates whether a PlayItem is a multi-angle, a “connection_condition”that indicates a connection state between the PlayItem (the currentPlayItem) and the PlayItem that is immediately previous to the currentPlayItem (previous PlayItem), a “ref_to_STC_id[°]” uniquely indicatingan STC Sequence targeted by the PlayItem, an “In_time” that is timeinformation indicating a start point of the playback section, an“Out_time” that is time information indicating an endpoint of theplayback section, an “UO_mask_table” indicating which user operation isto be masked in the PlayItem, an “STN_table”, a “BaseView_indicator”,and a “multi_clip_entry”. Among these, the items that are playback pathsare the pair of the “In_time” that is time information indicating astart point of the playback section, and the “Out_time” that is timeinformation indicating an endpoint of the playback section. Playbackpath information is composed of this pair of “In_time” and “Out_time”.

The STN_table (Stream Number_table) is a table in which pairs composedof a stream_entry, including a packet ID, and a stream attribute areallocated to a logical stream number. The order of the stream entry andstream attribute pairs in the STN_table indicates a priority ranking ofthe corresponding streams.

When a BaseView_indicator is 0, the BaseView is Left, and when theBaseView_indicator is 1, the BaseView is Right.

FIG. 25 depicts an internal structure of a SubPath information table.The lead line su1 shows a close-up of the internal structure of theSubPath information. As shown by the lead line su1, the SubPathinformation table includes a plurality of pieces of SubPath information1, 2, 3 . . . m. These pieces of SubPath information are a plurality ofinstances derived from one class structure and thus have a commoninternal structure. The lead line su2 shows a close-up of the sharedinternal structure of the SubPath information. As shown by the leadline, each piece of SubPath information includes a SubPath_typeindicating a SubPath type, and one or more pieces of SubPlayIteminformation ( . . . SubPlayItem information #1 to #m . . . ). The leadline su3 shows a close-up of the internal structure of the SubPlayItem.As shown by the lead line, each piece of SubPlayItem information iscomposed of a “Clip_information_file_name”, a “Clip_codec_identifier”, a“ref_to_STC_id[0]”, a “SubPlayItem_In_Time”, a “SubPlayItem_Out_time”, a“sync_PlayItem_id”, and a “sync_start_PTS_of_PlayItem”. The followingdescribes an internal structure of a SubPlayItem.

The “Clip_information_file_name” is information that uniquely specifies,by a file name of the clip information, a SubClip corresponding to aSubPlayItem.

The “Clip_codec_identifier” indicates an encoding method of the AV clip.

The “ref_to_STC_id[0]” uniquely indicates an STC Sequence targeted bythe SubPlayItem.

The “SubPlayItem_In_time” is information indicating a start point of theSubPlayItem on the playback time axis of a SubClip.

The “SubPlayItem_Out_time” is information indicating an end point of theSubPlayItem on the playback time axis of the SubClip.

The “sync_PlayItem_id” is information that uniquely identifies aPlayItem, from among the PlayItems in the MainPath, to be synchronizedwith the SubPlayItem. The SubPlayItem_In_time exists on the playbacktime axis of the PlayItem designated by the sync_PlayItem_id.

The “sync_start_PTS_of_PlayItem” indicates where, on a playback timeaxis of the PlayItem specified by the sync_PlayItem_id, a start point ofthe SubPlayItem specified by the “SubPlayItem” exists, with the timeaccuracy of 45 KHz.

FIG. 26 depicts playback sections defined for each of the left view andthe right view. FIG. 26 is drawn based on FIG. 23. More specifically,the In_Time and the Out_Time of a PlayItem is indicated in the time axison the second row of FIG. 23 on which FIG. 26 is based. On the time axisof the first row, the In_Time and Out_Time of a SubPlayItem is drawn.The third to fifth rows are identical to corresponding portions of FIG.23. The I pictures of the left view and the right view reside at thesame point on the time axis. The left view and the right view areassociated with each other by the PlayItem information and theSubPlayItem information.

The left view and the right view are associated with each other by thePlayItem information and the SubPlayItem information.

FIG. 27 shows an internal structure of the video stream table. As shownby the lead line mh1, the video stream number table includes pairs ofstream_entry and stream_attribute.

The “stream_entry” includes a “ref_to_stream_PID_of_main_Clip”indicating a reference value for the PIDs of the PES packets that makeup a primary elementary stream, a “video_format” indicating a videodisplay method such as NTSC, PAL etc., and a “frame_rate” indicating aframe rate such as 1/24 second, 1/29.94 second, etc.

FIG. 28 shows an internal structure of a PG stream information table inan STN_table. The PG stream information table in the STN_table isconstituted from “display method information” and “n pieces of streaminformation”. The n pieces of stream information are in one-to-onecorrespondence with stream numbers, and are constituted from astream_entry and a stream_attribute. The lead line gh1 shows a close-upof the internal structure of the stream_entry. One of“ref_to_stream_PID_of_mainClip”, “ref_to_Sub_Path_id”,“ref_to_SubClip_entry_id”, and “ref_to_stream_PID_of_subClip” is set inthe stream_entry. The “ref_to_stream_PID_of_SubClip” indicates a PID fora PG stream that corresponds to the stream number existing in the sameAV Clip as the video stream.

The following describes the extension data. FIG. 29 shows an internalstructure of extension data in the PlayList information. A lead line et1shows a close-up of the internal structure of the extension data. Asshown by this lead line, the extension data is composed of STN_tableextension fields corresponding respectively to the PlayItem informationpieces #1 to #N. The lead line et2 shows a close up of an internalstructure of an STN_table extension corresponding to the PlayIteminformation piece #1. As shown by this lead line, the STN_tableextension corresponding to the PlayItem information piece #1 includesthe “video stream number table”.

FIG. 30 depicts an internal structure of the video stream number table.

The video stream number table includes N enhanced_view_is_availableflags e1, and N pairs f1 of stream_entries and stream_attributes. Thesepairs are associated with the stream numbers 1 to N, and anenhanced_view_is_available flag can uniquely specify one of the pairswith use of a stream number from 1 to N. The pairs of stream_entry andstream_attribute can also be uniquely specified with use of a streamnumber from 1 to N.

As indicated with the lead line vh1, the “Stream_entry” includes a“ref_to_stream_PID_of_main_Clip” indicating the reference value for thePID assigned to the PES packets constituting the primary video stream.As indicated by the lead line vh2, the stream_attribute includes a“video_format” field and a “frame_rate” field.

The order of these stream_entries in these tables indicates the priorityorder of stream selection when the playback apparatus selects a stream.In other words, the playback apparatus preferentially selects thestreams whose entries are highest in the table.

When the “enhanced_view_is_available” flag is set to ON to makeEnhancedView valid, the “ref_to_stream_of_MainClip” is described toindicate the packet IDs 0x1011 and 0x1012.

The following describes the PG stream information table in the STN_tableextension. The internal structure of the PG stream information table inthe STN_table extension includes N pieces of stream information. The npieces of stream information are in one-to-one correspondence withstream numbers, and are constituted from a stream_entry and astream_attribute. The stream_entry includes a pair composed of areference value corresponding to a PID of a PES packet that comprises aplayable left view PG stream, and a reference value of a PID of a PESpacket comprising a right view PG stream.

Here, there are the same number of stream_entry and stream_attributepairs in the STN_table extension, as the number of stream_entry andstream_attributes pairs in the STN_table. The pairs of stream_entry andstream_attribute in the STN_table are in one-to-one correspondence withthe pairs of stream_entry and stream_attribute in the STN_table, andthese have the same stream numbers. The streams of left view and rightview indicated by the stream_entry of the STN_table_extention and thestreams shown by the stream_entry of the corresponding STN_table arebasically identical, and the display position is only different in thehorizontal direction. That is to say, a difference corresponding to theparallax between the human eyes has been added to the stream indicatedby the stream_entry of the STN_table, and these are the left view andright view streams indicated by the stream_entry of the STN_tableextention.

Note that the “content” referred to in the present specification is aunit including PlayList information managed by a certain title number,and a video stream in which AV clips referenced by the PlayList havebeen multiplexed, and is also called a “title”.

This concludes the description of the PlayList information.

Playback Apparatus

The following describes the details of the playback apparatus. FIG. 31shows the internal structure of the playback apparatus. The playbackapparatus 200 includes a BD drive 201, a system LSI (playback unit) 1,an HDMI interface 202, a playback state/setting register set 203, astatic scenario memory 205, a playback control engine 206, a heap memory207, a BD-J platform 208, a dynamic scenario memory 209, a modemanagement module 210, a command interpreter 211, and a UO detectionmodule 212.

The BD drive 201 performs loading/reading/ejection of the BD-ROM, andexecutes an access of the BD-ROM. As a specific structure, the BD-ROMdrive 112 includes, for example, a semiconductor laser (not depicted), acollimating lens (not depicted), a beam splitter (not depicted), anobjective lens (not depicted), a condenser lens (not depicted), and anoptical head (not depicted) including an optical detector (notdepicted). An optical beam emitted from the semiconductor laser isfocused onto an information side of an optical disk through thecollimating lens, the beam splitter, and the field lens. The focusedoptical beam is reflected/refracted on the optical disk and is focusedonto the light detector through the objective lens, the beam splitter,and the condenser lens. A signal generated according to the amount oflight received by the light detector represents data read from theBD-ROM.

As shown in FIG. 32, the system LSI (playback unit) 1 includes a readbuffer 2, a PID filter 3, transport buffers 4 a, 4 b, 4 c, 4 d, and 4 e,a peripheral circuit 4 f, a video decoder for left view 5 a, a videodecoder for right view 5 b, a video plane (L) 6 a, a video plane (R) 6b, a switch 7, a graphics decoder for left view 8 a, a graphics decoderfor right view 8 b, a graphics plane for left view 9 a, a graphics planefor right view 9 b, a CLUT UNIT for L 10 a, a CLUT UNIT for R 10 b, anOFFSET UNIT for L 11 a, an OFFSET UNIT for R 11 b, a CLUT output switch12, an adder 13, and an Audio Decoder 14.

The read buffer 2 is a typically an FIFO memory that temporarily storestherein a source packet read from a BD-ROM, and is a buffer fortransferring the source packet to the PID filter 3 after adjusting thetransfer speed thereof. Specifically, the read buffer 2 is composed of aright view read buffer for storing a right view source packet, and aleft view read buffer for storing a left view source packet.

The PID filter 3 performs filtering on a plurality of TS packets thatare output from the read buffer 2. The PID filter passes only TS packetshaving predetermined PIDs to one of the Transport Buffers 4 a, 4 b, 4 c,4 d, and 4 e. There is no buffering inside the PID filter 3.Accordingly, TS packets entering the PID filter 3 are instantaneouslywritten to one of the Transport Buffers 4 a, 4 b, 4 c, 4 d, and 4 e,according to the PID of the TS packet.

The Transport Buffers 4 a, 4 b, 4 c, 4 d, and 4 e are FIFO memories forstoring TS packets output from the PID filter 3. A speed at which a TSpacket is read from the Transport Buffers 4 a, 4 b, 4 c, 4 d, and 4 e isdenoted by transfer rate Rx.

The peripheral circuit 4 f has a wired logic for converting TS packetsread from the Transport Buffers 4 c and 4 d to functional segments. Thefunctional segments are then stored in the Coded Data Buffer (EB).

The video decoder for left view 5 a decodes TS packets output from thePID filter 3 to obtain an uncompressed left view picture, and writes thepicture to the Video Plane (L) 6 a.

The video decoder for right view 5 b decodes TS packets output from thePID filter 3 to obtain an uncompressed right view picture, and writesthe picture to the Video Plane (R) 6 b.

The video plane (L) 6 a is a memory for storing thereinuncompressed-type left view pictures.

The video plane (R) 6 b is a memory for storing thereinuncompressed-type right view pictures.

The switch 7 switches between outputting, to the adder 13, a left viewpicture and a right view picture.

The graphics decoder for left view 8 a decodes a left view PG stream toobtain uncompressed graphics, and writes the uncompressed graphics tothe graphics plane for left view 9 a as graphics objects. Specifically,as shown in FIG. 32, the graphics decoder for left view 8 a includes theCoded Data Buffer (EB) 81 a, a peripheral circuit 82 a, a StreamGraphics Processor 83 a, an Object Buffer (OB) 84 a, a CompositionBuffer (CB) 85 a, and a graphics controller 86 a.

The Coded Data Buffer (EB) 81 a is used for storing functional segmentstogether with DTSs and PTSs. Such functional segments are obtained byremoving a TS packet header and a PES packet header from each TS packetof a transport stream stored in the Transport Buffer 4 c and arrangingremaining payloads in sequence. DTSs and PTSs contained in the removedTS packet headers and PES packet headers are stored in the Coded DataBuffer in correspondence with the functional segments.

The peripheral circuit 82 a has a wired logic for transferring data fromthe Coded Data Buffer (EB) 81 a to the Stream Graphics Processor 83 aand transferring data from the Coded Data Buffer (EB) 81 a to theComposition Buffer 85 a. When the current time reaches a DTS of an ODS,the peripheral circuit transfers the ODS from the Coded Data Buffer (EB)81 a to the Stream Graphics Processor 83 a. Also, when the current timereaches the DTS of a PCS and a PDS, the peripheral circuit transfers thePCS and the PDS from the Coded Data Buffer (EB) 81 a to the CompositionBuffer (CB) 85 a.

The Stream Graphics Processor 83 a decodes the ODS to obtainuncompressed graphics having index colors, and transfers theuncompressed graphics to the Object Buffer (OB) 84 a as a GraphicsObject. The decoding by the Stream Graphics Processor 83 a starts at atime of a DTS correlated to the ODS, and ends by the ending time ofdecoding shown by the PTS correlated to the ODS. The decode rate Rd ofthe graphics object described above is the output rate of the StreamGraphics Processor 83 a.

The Object Buffer (OB) 84 a corresponds to a pixel buffer in ETSI EN 300743 standards. Graphics Objects decoded by the Stream Graphics Processor83 a are stored in the Object Buffer (OB) 84 a.

The Composition Buffer 85 a is a memory for storing a PCS and a PDS.

The Graphics Controller 86 a decodes the PCSs in the Composition Buffer85 a, and performs control based on the decoded PCSs. Specifically, thegraphics are written to the object buffer (OB) 84 a, the graphics areread from the object buffer (OB) 84 a, and the graphics are displayed.The execution timing of this control is based on the value of the PTS ofthe PES packet that stores the PCS.

The graphics decoder for right view 8 b decodes a right view PG streamto obtain uncompressed-type graphics, and writes the uncompressed-typegraphics to the graphics plane for right view 9 b as graphics objects.Since the details of the structure are the same as the graphics decoderfor left view 8 a, description thereof is omitted.

The graphics plane for left view 9 a is a plane memory having an areafor storing one screen worth of graphics, and stores therein one screenworth of uncompressed graphics (left view).

The graphics plane for right view 9 b is a plane memory having an areafor storing one screen worth of graphics, and stores therein one screenworth of uncompressed graphics (right view).

The CLUT UNIT for L10 a, with use of a color look-up table of a PDSindicated by a set palette_id, converts a pixel code stored in thegraphics plane for left view 9 a to a pixel value that is Y, Cr, Cb.

The CLUT UNIT for R10 b, with use of a color look-up table of a PDSindicated by a set palette_id, converts a pixel code stored in thegraphics plane for right view 9 b to a pixel value that is Y, Cr, Cb.

Here, the same palette_id is set in both the CLUT UNIT for L1 10 a andthe CLUT UNIT for R10 b.

The OFFSET UNIT for L11 a adjusts the depth of color-converted left-viewuncompressed graphics.

The OFFSET UNIT for R11 b adjusts the depth of color-convertedright-view uncompressed graphics.

The CLUT output switch 12 switches between left view uncompressedgraphics and right view uncompressed graphics when outputting theuncompressed graphics to be output to the adder 13.

The adder 13 multiplies the uncompressed graphics converted by the CLUTunit for L10 a (or the CLUT UNIT for R10 b), by a T value (transparency)shown in the PDS. The adder 13 then obtains a composite image by addingthe result of the multiplication to the uncompressed picture data storedin the Video Plane (L) 6 a (or the Video Plane (R) 6 b), and outputs thecomposite image.

The Audio Decoder 14 decodes TS packets output from the PID filter 3,and outputs uncompressed audio data.

This completes the description of the constituent elements of theplayback unit 1.

Returning to FIG. 31, the HDMI transmission/reception unit 202 includesan interface that is compliant, for example, with HDMI standards (HDMI:High Definition Multimedia Interface), and performs transmission andreception in a manner compliant with HDMI standards between the playbackapparatus and an apparatus (in the present example, the television 300),that is connected by HDMI. Video on which adding has been performed bythe adder 13, and uncompressed audio data that has been decoded by theaudio decoder 14, are transmitted to the television 300 via the HDMItransmission/reception unit 202.

The television 300 stores information indicating whether, for example,the television 103 supports stereoscopic display, information pertainingto possible resolution for monoscopic (i.e., two-dimensional) display,and information pertaining to possible resolution for stereoscopicdisplay. In response to a request from the playback apparatus via theHDMI transmission/reception unit 202, the television 103 transmits tothe playback apparatus requested necessary information (for example,information pertaining to whether the television 300 supportsstereoscopic display, information pertaining to possible resolution formonoscopic (i.e., two-dimensional) display, and information pertainingto possible resolution for stereoscopic display). In this way, theplayback apparatus is enabled to acquire information indicating whetherthe television 103 supports stereoscopic display, from the television300 via the HDMI transmission/reception unit 202.

The playback apparatus/setting register (PSR) set 203 is a set ofregisters including a player status register storing the playback statusof a PlayList, a playback setting register storing configurationinformation indicating the configuration of the playback apparatus, ageneral purpose register available for storing arbitrary informationused by the contents. The playback status of a PlayList representsinformation such as a type of AV data that is being used from amongvarious types of AV data information indicated in the PlayList, and acurrent position (time) of the PlayList is being played back. When theplayer status of the PlayList changes, the playback control engine 206stores the changed player status in the PSR set 203. Further, inresponse to a request from an application being executed by the commandinterpreter, which is the executing entity in the HDMV mode, or by theJava platform, which is the executing entity in the BD-J mode, theplayback control engine stores the value specified by the application inthe PSR set 203 and passes the stored value to the application.

Also, there are flags, etc. indicating stereoscopic playback capabilityand stereoscopic playback display method, for example, in the PSR set203.

Stereoscopic capability indicates whether a playback apparatus has theability to execute stereoscopic playback.

A stereoscopic playback flag indicates whether a user intends to executestereoscopic playback.

The static scenario memory 205 is a memory for storing current PlayListinformation and current clip information. The current PlayListinformation refers to a specific piece of PlayList information targetedfor current processing from among the plurality of PlayList informationpieces that can be accessed from the BD-ROM. The current clipinformation refers to information targeted for current processing fromamong the plurality of PlayList information pieces that can be accessedfrom the BD-ROM.

The playback control engine 206 executes AV playback functions andPlayList playback functions in accordance with function calls from thecommand interpreter, which is an execution entity in the HDMV mode, orfrom the Java platform, which is the execution entity in the BD-J mode.The AV playback functions are functional operations that follow DVDplayer and CD player operations, and include the following types ofprocessing: Playback Start, Playback Stop, Pause, Stop Pause, StopStatic Image Function, Fast Forward at a speed specified with animmediate value, Rewind at a speed specified with an immediate value,Audio Change, SubTitle Change, and Angle Change. The PlayList playbackfunctions are the functions Playback Start and Playback Stop, among theAV playback functions listed above that are performed according toPlayList information.

The heap memory 207 is a stack area for storing bytecodes of systemapplications, bytecodes of BD-J applications, system parameters used bysystem applications, and application parameters used by BD-Japplications.

The BD-J platform 208 is a Java platform that is the execution entity inthe BD-J mode, and fully implements Java2 Micro Edition (J2ME) PersonalBasis Profile (PBP 1.0) and Globally Executable MHP specification(GEM1.0.2) for package media targets. The BD-J platform 208 includes aclass loader and a bytecode interpreter. The class loader is one ofsystem applications and reading the bytecode of a BD-R application froma class file contained in a JAR archive file into the heap memory, sothat the BD-J application is loaded. The bytecode interpreter convertsthe bytecode comprising the BD-J application stored in the heap memory207 and the bytecode comprising the system application into native codeand enables the MPU to execute the BD-J application.

The dynamic scenario memory 209 is a memory for storing a currentdynamic scenario and used for the processing by the command interpreter,which is the execution entity in the HDMV mode, and by the Javaplatform, which is the execution entity in the BD-J mode. The currentdynamic scenario refers to a scenario that is targeted for currentexecution from among an Index.bdmv, a BD-J object, and a movie objectrecorded on the BD-ROM.

The module manager 210 that is an exemplary mode management module holdsthe index.bdmv read out from the BD-ROM, and performs mode managementand branch control. The mode management by the module manager 210 is anassignment of a module, namely, specifying which of the commandinterpreter 211 and the BD-J module executes the dynamic scenario.

The command interpreter 211 is an exemplary HDMV module and is a DVDvirtual player, which is the execution entity in the HDMV mode. Thecommand interpreter, which is the execution entity in the HDMV mode,decodes and executes a navigation_command included in a scenarioprogram. Since the navigation_commands are written in a syntax that issimilar to DVD-Video, playback control similar to DVD-Video can beexecuted by executing the navigation_command.

The UO detection module 212 detects a user operation made on the remotecontroller 500 or on the front panel of the playback apparatus 200 andoutputs information indicative of the detected user operation(hereinafter, simply “UO: User Operation) to the mode management module210. From among the received UOs, UOs compliant with the current mode ofthe playback apparatus are selectably passed to one of the modules thatexecutes the current mode. For example, if user operations for moving acursor in one of up, down, right, and left directions and for activatingthe current selection is received in the HDMV mode, the UO detectionmodule 212 passes the UOs to the HDMV mode module.

PlayList Playback Processing

The following describes the details of the PlayList playback processing.

FIG. 33 is a flowchart showing a processing procedure for PlayListplayback processing.

In Step S1, PlayList information is read. Subsequently Steps S2-S5 areperformed. In Step S2, it is judged whether the playback apparatus hasthe capability. In Step S3, it is judged whether a television connectedto the playback apparatus has the capability of stereoscopic playback.In Step S4, it is judged whether the display method flag included in theplayback attribute information of the current PlayList is valid. If anyof the judgments in Steps S2-S4 result in “No”, Step S5 is performednext to execute playback of each PlayItem based on the STN_tableincluded in the respective pieces of PlayItem information.

If all the judgments in Steps S2-S4 result in “YES”, Step S5 isperformed to execute each PlayItem based on the STN_table_extensionincluded in the respective pieces of PlayItem information.

FIG. 34 is a flowchart showing a procedure for executing playbackaccording to STN_table_extensions.

In Step S51, the current PlayItem number is initialed to “1” and theprocessing goes onto the loop composed of Steps S52-S62. In the loop,Steps S52-S60 are executed on the current PlayItem number and then thecurrent PlayItem number is incremented (Step S61) to repeat the StepsS52-S60 on the new current PlayItem number. The loop is repeated untilthe current PlayItem number reaches the last one of the PlayItem numbers(Step S62: Yes). The details of Steps S52-S60 are described below.

In Step S52, the PlayItem.In_Time and PlayItem.Out_Time of the currentPlayItem are converted into Start_SPN[i] and End_SPN[i], with the use ofthe entry map corresponding to the packet ID of the BaseView stream.

Next, a suitable EnhancedView stream is selected and a suitable currentPG stream is selected (Step S53). The current stream numbers of theselected streams are stored to the PSRs (Step S54). A SubPlayItemassociated with the current stream number is specified (Step S55). TheSubPlayItemIn_Time and SubPlayItemOut_Time are converted to Start_SPN[j]and End_SPN[j], with the use of the entry map [j] corresponding to thepacket ID [j] of the EnhancedView stream (Step S56).

In order to read the TS packet[i] having the packet ID [i], the extentsbelonging to the read range [i] from the Start_SPN[i] to End_SPN[i] arespecified (Step S57). In addition, in order to read TS packet [j] havingthe packet ID [j], the extents belonging to the read range fromStart_SPN[j] and End_SPN[j] are specified (Step S58). Next, in Step S59,the extents belonging to the respective read ranges [i] and [j] aresorted in ascending order of the addresses. In Step S60, an instructionis given to the drive to sequentially read the extents belonging to therespective read ranges [i] and [j] based on the sorted addresses. Thisconcludes the description of the playback procedure executed based onthe STN_table_extension.

The following describes the procedure for selecting a PG stream.

There are two types of procedure for selecting a PG stream based on theSTN_table or STN_table_extension. One is “Procedure when playbackcondition is changed” and the other is “Procedure when Stream change isrequested”. The selection of the PG stream is performed by the playbackcontrol engine 206.

The “Procedure when playback condition is changed” is executed when thestatus of the playback apparatus changes in response to an eventoccurred to the playback apparatus.

The “Procedure when Stream Change is requested” is executed when theuser requests the switching between streams.

FIG. 35A is a flowchart showing the procedure for setting the PSR 2 atthe time when a status change takes place. Here, the PSR2 indicates thecurrent stream number of the PG.

In Step S11, a judgment is made as to whether the number of entriesincluded in the STN_table is equal to “0”. If the number of entries isequal to “0”, the value held in the PSR 2 is maintained (Step S13).

Step S12 is performed if the number of entries included in the STN_tableis not equal to “0”. In Step S12, a judgment is made as to whether thenumber of entries included in the STN_table is larger than the valueheld in the PSR 2 and if the condition (A) is satisfied. The condition(A) is satisfied if the playback apparatus is capable of executingplayback of the PG stream specified by the value held in the PSR 2. Ifthe judgment in Step S12 results in “Yes”, the value held in the PSR 2is maintained (Step S14). On the other hand, if the value held in thePSR 2 is larger than the number of entries included in the STN_table orthe condition (A) is not satisfied, the value of PSR 2 is reset (StepS15).

FIG. 35B is a flowchart showing the procedure for setting the PSR 2 atthe time when a stream change takes place. The flowchart shown in FIG.42B is identical to the flowchart shown in FIG. 42A, except that thedenotation of “PSR 2” is replaced with X. The “X” denotes a value basedon a user operation.

In Step S20 shown in this flowchart, a judgment is made as to whetherthe number of entries included in the STN_table is larger than X and ifthe condition (A) is true. The condition (A) is true if the playbackapparatus is capable of executing playback of the PG stream specified bythe value held in the PSR 2. This judgment is made by comparing thevalue of PSR 15 (indicating a subtitle capability) with theStream_coding_type of the PG stream. If X satisfies this condition, thevalue of the PSR 2 is set to the value of X (Step S21).

On the other hand, if the value of X is larger than the number ofentries or the condition (A) is not satisfied, it is then judged if thevalue of X is equal to “0xFFFF” (Step S22). If the value of X is notequal to “0xFFFF”, the PG stream number specified by the user is assumedto be invalid. Thus, the value of X specified by the user operation isdiscarded and the value held in the PSR 2 is maintained (Step S24). Ifthe value held in the PSR 2 is equal to “0xFFFF”, the PSR 2 is setaccordingly (Step S23).

FIG. 36 is a flowchart indicating a PSR2 setting procedure. In theflowchart, Steps S31 and S32 form a loop in which Steps S33 to S35 arerepeated for each subtitle stream described in the STN_table. In theloop, the PG stream currently being processed is referred to as a PGstream i. In Step S33, it is judged whether the PG stream i is agraphics subtitle stream or a text subtitle stream. If the subtitlestream i is a graphics subtitle stream, the processing proceeds to StepS34.

In Step S34, it is judged whether the graphics subtitle stream isatisfies the following conditions (a) and (b).

The condition (a) is satisfied if the playback apparatus is capable ofexecuting playback of the graphics subtitle stream i.

The condition (b) is satisfied if the language attribute of the graphicssubtitle stream i matches the language settings of the playbackapparatus.

The judgment regarding the condition (b) is made by checking if thePG_language_code included in the STN_table matches the value held in thePSR 16 (indicating the language setting of the apparatus).

In Step S35, on the other hand, it is judged whether the text subtitlestream satisfies the following conditions (a) and (b).

The condition (a) is satisfied if the playback apparatus is capable ofexpanding the text subtitle stream i into fonts.

The condition (b) is satisfied if the language attribute of the graphicssubtitle stream i matches the language settings of the playbackapparatus.

The judgment regarding the condition (a) is made by checking whether thePSR 30 of the playback apparatus indicates that the playback apparatusis “playback capable”. The judgment regarding the condition (b) is madeby checking if the PG_language_code included in the STN_table matchesthe value held in the PSR 16.

When the processes of Steps S33 to S35 have been repeated on each PGstream, the processes of Steps S36-S41 are performed.

In Step S36, a judgment is made as to whether any graphics subtitlesatisfies the condition (a). If no graphics subtitle satisfies thecondition (a), the PSR 2 is set to an invalid value (0xFFFF) (Step S38).

In Step S37, a judgment is made as to whether any PG stream satisfiesboth the conditions (a) and (b). If one or more PG streams satisfy boththe conditions (a) and (b), the PSR 2 is set to a value equal to thestream number of one of the PG streams having a higher entry order inthe STN_table (Step S41).

In Step 39, the PSR 2 is set to a value equal to the stream number ofone of the streams having the highest entry order in the STN_table, fromamong graphics subtitle streams and text subtitle streams eachsatisfying the condition (a) only.

When the PRS2 has been set, a judgment is made as to whether theplayback apparatus is capable of performing 3D playback using the leftview and the right view. If 3D playback is possible, PIDs for a leftview PG stream and a right view PG stream corresponding to a PSR2 streamnumber are set in the PID filter 3, and packet filtering is performed.

The left view graphics decoder and the right view graphics decoder arestarted up, and decoding is performed on two systems of TS packetgroups.

According to the embodiment described above, the plurality of DSs of theleft view PG stream and the plurality of DSs of the right view PG streamare in one-to-one correspondence, and an identical PTS is set in thepresentation composition segments of corresponding DSs. Also, acomposition_state having identical content is set in the presentationcomposition segments of corresponding DSs.

Therefore, if the composition_state is, for example, Epoch Start for theDS of one of the PG streams, the composition_state for the DS of theother one of the PG streams is also Epoch Start.

Accordingly, since these DSs are provided at points likely to be chosenas playback start points, during random access, if decoding one of thePG streams is possible, decoding the other PG stream is necessarilypossible. Therefore, this enables ensuring the stereoscopic view ofgraphics during random access.

Also, in the DSs corresponding to the left view PG stream and the rightview PG stream, the composition_number and the forced_on flag are setwith identical content, and the PDS of the corresponding DS is set withidentical content. Furthermore, in the WDS, the window_width,window_height, and window_vertical_position are set with identicalcontent, and in the PCS, the object_cropping_width, theobject_cropping_height, the object_cropping_vertical_position, and theobject_vertical_position are set with identical content.

In this way, in the DS corresponding to the left view PG stream and theright view PG stream, fields other than the horizontal positions of theobject and the display position(window_horizontal_position,object_horizontal_position, object_cropping_horizontal_position) are setwith identical content, thus enabling the control content of the twographics controllers in the playback apparatus to be substantiallysimilar, and making the hardware structure simpler.

Variation 1-1

The playback apparatus includes a CLUT UNIT for L10 a for left view anda CLUT UNIT for R10 b for right view. Since the same palette_id is to beset for each CLUT UNIT, the same CLUT UNIT can be used. The followingdescribes a variation in which left view and right view use the sameCLUT UNIT.

FIG. 37 shows an internal structure of a system LSI in the playbackapparatus in variation 1-1.

The Graphics Plane for Left view and the Graphics Plane for Right view 9b are both connected to the CLUT output switch 15.

When outputting the uncompressed graphics to be output to the CLUT UNIT10, the CLUT output switch 15 switches between left view uncompressedgraphics and right view uncompressed graphics.

The OFFSET UNIT 11 adjusts the depth of the uncompressed graphics of theleft view or the right view that have been color-converted by the CLUTUNIT 10, and outputs the depth to the adder 13.

As described above, since the output of the Graphics Plane for Left view9 a and the Graphics Plane for Right view 9 b both connect to the sameCLUT UNIT 10 via the CLUT output switch 15, along with reducing thescale of the hardware, there is a benefit that a 3D graphics decoder canbe formed from the 2D graphics decoder with few changes.

Embodiment 2

Although embodiment 1 describes realizing a stereoscopic view of a PGstream in a 2-decoder model, the present embodiment describes astereoscopic view of interactive graphics (hereinafter referred to as“IG”) in a 2-decoder model.

An IG stream realizes interactive display of a graphics object. FIG. 38Ashows the structure of the IG stream. The first row shows a TS packetsequence that makes up an AV clip. The second row shows a PES packetsequence that makes up a graphics stream. The PES packet sequence in thesecond row is constituted from TS packets from which payloads have beenremoved for TS packets having a predetermined PID, from among the TSpackets in the first row, and linking the payloads together. The thirdrow shows a structure of a graphics stream. The graphics stream iscomposed of functional segments referred to as an ICS (InteractiveComposition Segment), a PDS (Palette Definition Segment), an ODS (ObjectDefinition Segment), and an END (END of Display Set Segment). ICS isreferred to as a presentation composition segment, and controls screenstructure of interactive graphics objects. Since the correspondencebetween PES packets and functional segments, and details of otherfunctional segments, are the same as PG, description thereof is omitted.

FIG. 38B shows a PES packet obtained by converting a functional segment.Since FIG. 38B is also similar to the case of PG, description thereof isomitted here.

FIG. 39 shows logical structures of various types of functionalsegments. The first row of FIG. 39 shows Epoch, the second row showsDisplay Set, the third row shows types of Display Set, and the fourthrow shows functional segments belonging to a Display Set. There are alsothe types Epoch Start, Acquisition Point, and Normal Case, but these arethe same as described in the case of PG.

The IG stream is characterized by control of behavior of the multi-pagemenu in accordance with progress, along the above-described playbacktime axis, of moving picture playback. The composition that allows thischaracteristic to be realized relates to an interactive_composition inthe ICS. The following describes the internal structure of theinteractive_composition.

FIGS. 40A and 40B show the correspondence between the ICS and theinteractive_composition. The relationship between the ICS and theinteractive_composition is either a one-to-one correspondence of thetype shown in FIG. 40A or a one-to-many correspondence of the type shownin FIG. 40B.

The relationship is a one-to-one correspondence when a data size of theinteractive_composition is small enough to be stored in a single ICS.

The relationship is a one-to-many correspondence when a data size of theinteractive_composition is large, and the interactive_composition isfragmented and stored by a plurality of ICS. Since theinteractive_composition can be stored using a plurality of ICS, there isno limit to its data size, which may be increased to, say, 512 kMB, 1MB, or more. A one-to-many correspondence is also possible between theICS and the interactive_composition. However, for simplicity, the casewhere the ICS and the interactive_composition are in one-to-onecorrespondence is used in the following description.

FIG. 41 shows a data structure of an ICS. Each ICS stores either anentire interactive_composition or a portion of aninteractive_composition obtained by fragmentation. As shown in the lefthand side of FIG. 41, the ICS includes a segment_descriptor to indicatethat it is an ICS, horizontal and vertical pixel numbers to be used bythe ICS, a video descriptor indicating a frame rate, acomposition_descriptor, and an interactive_composition or a portion ofan interactive_composition_data_fragment obtained by fragmentation.

The arrow cu1 in FIG. 41 points to a close-up of a data structure of thecomposition_descriptor. As indicated, the composition_descriptorincludes a composition_state field that indicates whether a Display Setto which the ICS belongs is of the Normal Case-type, the AcquisitionPoint-type, the Epoch Start-type, or Effect_Sequence-type, and acomposition_number indicating a count of screen combinations.

The arrow cut of FIG. 41 points to a close-up of a data structure of theinteractive_composition. As indicated, the interactive_compositionincludes an interactive_composition_length_field, a stream_model_field,a user_interface_model_field, a composition_time_out_pts, aselection_time_out_pts, a user_time_out_duration, and pieces of pageinformation (1) (2) . . . (i) . . . (number_of_page−1), each of whichcorresponds to a displayable page in the multi-page menu.

The interactive_composition_length field indicates the data length ofthe interactive_composition.

The stream_model field indicates a type of stream model theinteractive_composition is using. The stream model indicates the formatused to record the interactive_composition on the BD-ROM, and the mannerin which the interactive_composition is to be managed by the compositionbuffer in the reproduction device. Specifically, the stream modelindicates whether the graphics stream i) has been de-multiplexed from anAV Clip and loaded in the composition buffer or ii) has been preloadedin the composition buffer as a SubClip.

The user_interface_model field indicates whether the user type of userinterface assumed by the interactive_composition is an Always-on U/I ora Pop-up U/I. With an Always-on U/I a menu display/deletion occurs inaccordance with playback progress of the AV Clip. With a Pop-up U/I, amenu display/deletion is triggered by a user operation.

The composition_time_out_pts indicates an end period (Epoch END) of theEpoch to which the ICS belongs. Since interactive control by means ofthe ICS becomes impossible at the end period, the time point indicatedby the composition_time_out_pts indicates the end of interactivecontrol.

The selection_time_out_pts indicates a time period allowed for a buttonin a selected state to automatically activate. More specifically, eachbutton corresponds to an item selection on the multi-page menu, and theselection_time_out_pts specifies the time taken to alter the button toan active state upon selection.

The IF statement in FIG. 41 (if (Stream_model==‘0b’)) shows that theabove-described composition_time_out_pts and selection_time_out_pts areinformation elements that appear only when the stream_modelfield=multiplexed. If the stream model is preloaded, neither thecomposition_time_out field nor the selection_time_out field are used.

The user_time_out_duration field indicates a period of time allowedbefore deletion for a page displayed in response to a user operation. Inthe case of the Always-on U/I, since only pages from the second pageonwards are displayed in response to user operations, only theseSubPages are deleted when the user_time_out duration has elapsed. Afterthe SubPages are deleted, the first page is displayed. In the case ofthe Pop-up U/I, on the other hand, since the every page of themulti-page menu is displayed in response to a user operation, all pagesare deleted when the user_time_out_duration has elapsed. The result is astate in which nothing is displayed (No Menu Display).

FIG. 42 shows an internal structure of page information for a given page(page (x)) among a plurality of pages. As indicated on the left-handside of FIG. 42, page information (x) includes a “page_id” that uniquelyidentifies page (x), a “page_version_number”, a “UO_mask_table”, an“in_effect” field, indicating a display result to be played back at thedisplay start time of the page (x), an “out_effect” field indicating adisplay result to be played back at the display end time of the page(x), an “animation_frame_rate_code” in which a frame rate to be appliedis written, a “default_selected_button_id_ref”, a“default_selected_button_id_ref”, a “default_activated_button_id_ref”, a“pallet_id_ref”, and a “button information (1) (2) . . .(number_of_buttons−1)” corresponding respectively to the plurality ofbuttons on the page (x).

The “UO_Mask_Table” field indicates whether to permit or not permit auser operation on the page (x). If this mask field is set to not permit,any user operation on the playback apparatus is invalidated.

The “default_selected_button_id_ref” field indicates, when display ofpage (x) begins, whether buttons whose default state is the selectedstate are determined interactively or passively. Specifically, when thefield contains 0xff, the buttons whose default state is the selectedstate are determined interactively. In this case, settings values in thePlayer Status Register (PSR) in the reproduction device arepreferentially interpreted, and the buttons indicated in the PSR are setto the selected state. When the field does not contain 0xff, the buttonswhose default state is the selected state are determined statically. Inthis case, the button numbers specified in thedefault_select_button_id_ref field are written over the PSR, and thecorresponding buttons are set to the selected state.

The “default_activated_button_id_ref” field indicates buttonsautomatically set in the active state when a time point indicated by theselection_time_out_pts is reached.

If the default_activated_button_id_ref field contains FF, the button inthe selected state for the predetermined period is automaticallyselected. If the default_activated_button_id_ref field contains 00, thebutton in the selected state is not automatically selected. When thedefault_activated_button_id_ref field contains a value other than 00 orFF, the value is interpreted as a valid button number.

The “palette_id_ref” field indicates the ID of the palette to be set inthe CLUT unit for the page(x).

The button information (button_info field) defines each button displayedon the page(x). The above-described fields specify each page in themulti-page menu. The following describes the internal structure of thebutton information. In the description, an arbitrary one of the buttonson the page (x) is denoted as a button (i). In FIG. 42, arrows cx1indicate a close-up of the internal structure of the button_info(i).

Each button presented on the page has three states which area normalstate, a selected state, and an activated state.

When a button is in the normal state, the button is simply displayed.When a button is in the selected stated, the button is currently focusedas a result of a user operation but not yet activated. When a button isin the activated state, the button has been activated. Since each buttonhas these three states, the button_info (i) is provided with thefollowing information.

The button_id field shows a value uniquely identifying the button(i) inthe interactive_composition structure.

The button_numeric_select_value field shows a flag indicating whetherthe button(i) is numerically selectable.

The auto_action_flag field shows whether the button (i) is automaticallyactivated. When the auto_action_flag field is set to ON (bit value of“1”), the button(i) is transferred not to the selected state butdirectly to the activated state when selected. On the other hand, whenthe auto_action_flag field is set to OFF (bit value of “0”), thebutton(i) is transferred not to the activated state but to the selectedstate when selected.

The button_horizontal_position field and button_vertical_position_fieldrespectively specify the horizontal and vertical positions of the topleft pixel of the button(i) on the interactive display.

The neighbor_info structure shows buttons to receive the selected statein response to user operations made to move a focus in an up, down,right, and left direction when the button(i) is in the selected state.The neighbor_info structure is composed of upper_button_id_ref,lower_button_id_ref, left_button_id_ref, and Right_button_id_ref fields.

The upper_button_id_ref field specifies the button number of a button toreceive the selected state if a user operation instructing to move thefocus in the up direction is made while the button(i) is in the selectedstate. The move up operation is executed at a push of a Move Up key ofthe remote controller. If this field is set to the same button number asthat of the button(i), the user operation to the Move Up key is ignored.

Similarly, the lower_button_id_ref field, the left_button_id_ref field,the Right_button_id_ref field each specify the button numbers of abutton to receive the selected state if a user operation instructing tomove the focus to the down, left, or right direction is made, while thebutton(i) is in the selected state. The respective operations areexecuted at a push of the Move Down key, Move Left key, and Move Rightkey of the remote controller. If these fields are set to the same buttonnumber as that of the button (i), the user operations to the those keysare ignored.

The normal_state_info structure defines the normal state of thebutton(i), and is composed of the normal_start_object_id_ref,normal_end_object_id_ref, and normal_repeat_flag fields.

The normal_start_object_id_ref field specifies the first one of theobject_id values that are serially assigned to a sequence of ODSs usedto present the normal state of button(i) in animation.

The normal_end_object_id_ref field specifies the last one of theobject_id values that are serially assigned to the sequence of ODSs usedto present the normal state of button(i) in animation. If thenormal_end_object_id_ref field specifies the same ID value as that ofthe normal_start_object_id_ref, the static image of a graphics objectidentified by that ID is presented as the button(i).

The normal_repeat_flag field specifies whether the animation of thebutton(i) in the normal state is to be continuously repeated.

The selected_state_info structure defines the selected state of thebutton(i). The selected_state_info structure is composed of theselected_state_sound_id_ref, selected_start_object_id_ref,selected_end_object_id_ref, and selected_repeat_flag fields.

The selected_state_sound_id_ref field specifies sound data to bereproduced as a click sound when the button(i) is put to the selectedstate. The sound data is specified by the sound_id of a piece of sounddata contained in the file called sound.bdmv. When this field is set to“0xFF”, no sound data is associated with the selected state of thebutton(i) and thus no click sound is reproduced when the button(i) ischanged to the selected state.

The selected_start_object_id_ref field specifies the first one of theobject_id values that are serially assigned to a sequence of ODSs usedto present the selected state of button(i) in animation.

The selected_end_object_id_ref field specifies the last one of theobject_id values that are serially assigned to the sequence of ODSs usedto present the normal state of button(i) in animation. If theselected_end_object_id_ref field specifies the same ID value as that ofthe selected_start_object_id_ref, the static image of a graphics objectidentified by the ID is presented as the button(i).

The selected_repeat_flag field specifies whether the animation of thebutton(i) in the selected state is to be continuously repeated. If theselected_start_object_id_ref and selected_end_object_id_ref fields havethe same value, the selected_repeat_flag field is set to the value of“00”.

The activated_state_info structure defines the activated state of thebutton(i), and is composed of the activated_state_sound_id_ref,activated_start_object_id_ref, and activated end object_id_ref fields.

The activated_state_sound_id_ref field specifies sound data to bereproduced as a click sound when the button (i) is put to the activatedstate. The sound data is specified by the sound_id of a piece of sounddata contained in the sound.bdmv file. When this field is set to “0xFF”,no sound data is associated with the activated state of the button(i)and thus no click sound is reproduced when the button(i) is changed tothe activated state.

The activated_start_object_id_ref field specifies the first one of theobject_id values that are serially assigned to a sequence of ODSs usedto present the activated state of button(i) in animation.

The activated end_object_id_ref field specifies the last one of theobject_id values that are serially assigned to the sequence of ODSs usedto present the activated state of button(i) in animation.

The “navigation_command” structure shows a navigation command to beexecuted when the button(i) is activated. A representative example ofthe navigation_command is a SetButtonPage command. The SetButtonPagecommand instructs the playback apparatus to display a desired page ofthe Multi-Page menu with a desired one of buttons on the page in theselected state. With the use of such navigation commands, contentcreators can readily describe page transitions at the time of authoring.

This completes the description of the internal structure of the buttoninformation.

Restrictions when Using the 2-Decoder Model

The following describes how the fields of the above-described functionalsegments are set for left view and right view, respectively, and this isbasically is the same as in the case of PG. That is to say, the type ofDS and the composition_number in the left view IG stream are set withidentical values as the type of DS and the composition_number in thecorresponding right view IG stream.

Also, an identical value is set in the interactive composition segmentin the left view DS and the interactive composition segment in thecorresponding right view DS for the PTS of the PES packets that storethese presentation composition segments.

Furthermore, identical content is set for correspondences between codevalue, luminance and color difference, for both the palette definitionsegment of the left view DS, and the palette definition segment of thecorresponding right view DS, and the palette_id is also identical.

The following describes how fields other than those described above arewritten in ICS. FIGS. 43 and 44 show an example of writing ICS thatbelongs to a Display Set. FIGS. 43 and 44 show an example in which aplurality of buttons are written in a graphics plane.

FIG. 43 shows an example of an ICS belonging to a left-view DS. As shownin FIG. 43, a title 1 button defined by button information (0) isprovided in a range which takes the button_horizontal_position 1 (left)and the button_vertical_position as a basepoint (top left) in thecoordinate system of the graphics plane. Also, a title m button definedby button information (number_of_buttons−1) is provided in a range whichtakes the button_horizontal_position m (left) and thebutton_vertical_position as a basepoint (top left). In this way, thetitle buttons are written in the graphics plane. This causes the titlebutton to be displayed as composited with the moving picture.

FIG. 44 shows an example of an ICS belonging to a right-view DS. Asshown in FIG. 44, a title 1 button defined by button information (0) isprovided in a range which takes the button_horizontal_position 1 (right)and the button_vertical_position as a basepoint (top left) in thecoordinate system of the graphics plane. Also, a title m button definedby button information (number_of_buttons−1) is provided in a range whichtakes the button_horizontal_position m (right) and thebutton_vertical_position as a basepoint (top left). In this way, thegraphics are written in the graphics plane. This causes the graphics tobe displayed as composited with the moving picture.

Although the button_vertical_position is set with the same value as abasepoint in both the left view and the right view as described above,the button_horizontal_position is set with different values in the leftview and the right view.

Also, the button_id is set with the same value in both the left view andthe right view. The reason for this is that a playback apparatus thatuses 2 decoders has a structure in which the two decoders are sharedbetween the graphics controller. For this reason, unless the button_idis set with the same value as the graphics object of the DScorresponding to the left view and the right view, the graphicscontroller cannot distinguish which graphics objects correspond to eachother.

The items other than the above in the button_information, namely theneighbor_info, the normal_state_info, the selected_state_info, theactivated_state_info, and the navigation_command, identical content isused for both the left view and the right view.

Furthermore, in the DSs corresponding to the left view and the rightview, identical values are set for the composition_time_out_pts, theselection_time_out_pts, and the user_time_out_duration.

Adding the above restrictions enables realizing stereoscopic view in a2-decoder structure by a left view IG stream and a right view IG stream.

Playback Apparatus

The following describes the details of the system LSI included in theplayback apparatus of the present embodiment. FIG. 45 shows the internalstructure of the system. LSI in the present embodiment. The basicstructure is the same as FIG. 32. The difference from FIG. 32 is thatthe graphics controller 86 is shared between the left view graphicscontroller and the right view graphics controller. The followingdescribes the reason for this. In the case of IG, the graphics decoderis expected to receive a notification of UO from a UO detection module.For this reason, when there are two graphics controllers, each receive aUO notification from the UO detection module. If this is done, sincenotifications of the UO from the UO detection module to the graphicscontroller can only be performed serially, a time difference occursbetween the graphics controllers. As a result, when executing processingbased on UO, there is a possibility of a malfunction occurring betweenthe graphics controllers. For this reason, the graphics controller 86 isshared between the left view and the right view.

The graphics controller 86 that is shared between the left view and theright view decodes an ICS provided in a Composition Buffer 85 a,performs control based on the ICS, decodes the ICS provided in thecomposition buffer 85 b, and performs control based on the ICS.

Also, when notification of a UO is received from the UO detection module212, the graphics controller 86 performs control corresponding to thisUO on the Graphics Decoder for Left view and the Graphics Decoder forRight view, respectively.

In more detail, the Graphics Controller 86 performs drawing of graphicswith reference to button information specified by the PSR 11 (currentpage information) from among the plurality of page information pieces inthe ICS provided in the Composition Buffer 85 a. This drawing isperformed by reading graphics specified by the normal_start_object_idand normal_end_object_id of the normal_state_info from the Object Buffer84 a, in the button information pieces in the current page information,and writing the graphics to the Graphics Plane for Left view 9 a. Amongthe button information of the current page information, the informationspecified by the PSR 10 is the graphics specified by theselected_start_object_id and the selected_end_object_id of theselected_state_info that is read from the Object Buffer 84 a, and iswritten to the Graphics Plane for Left view 9 a. According to thedrawing, the pages arranged from the title 1 button to the title mbutton appear in the Graphics Plane for Left view 9 a, and arecomposited with the moving picture. The right view is also similar.

According to the embodiment as described above, in the DS thatcorrelates the left view IG stream and the right view IG stream, sincethe composition_state is set with identical content, by providing theseDSs at points likely to be chosen as playback start points, similarly tothe case of PS, enables ensuring the stereoscopic view of graphicsduring random access.

Also, by causing graphics controller in the playback apparatus to beshared between the left view and the right view, when compared to a caseof using a 2-controller structure, enables reducing the scale ofhardware, and realizing a reduced cost of the apparatus.

Supplementary Remarks

Although this completes the description of the best embodiment of thepresent invention known to the applicant at the time of filing thepresent application, of course, the present invention is not limited tothe above embodiment.

(1) The playback apparatus 200 may further include a local storage. Thelocal storage includes a built-in medium and a removable medium and isused for storing data, such as downloaded additional content and datathat is used by applications. The storage area for additional contentsis divided specifically for individual BD-ROMs, and the storage area forstoring data used by applications is divided specifically for individualapplications. Merge management information specifying rules for merginga downloaded additional content with data on a BD-ROM is also stored inthe built-in medium or the removable medium.

The built-in medium is, for example, a writable recording medium such asa hard disk drive or a memory built into the playback apparatus.

The removable medium is, for example, a portable recording medium, andpreferably is a portable semiconductor memory card such as an SD card.

The following description is given by way of an example in which theremovable medium is a semiconductor memory card. The playback apparatusincludes a slot (not depicted) for inserting the removable medium, andan interface (for example a memory card I/F) for reading the removablemedium inserted in the slot. When the semiconductor memory card isinserted in the slot, the removable medium is electrically connected tothe playback apparatus, and with use of an interface (for example, thememory card I/F), the data recorded on the semiconductor memory card canbe converted to electrical signals and read.

When the playback apparatus 200 includes a local storage, a virtual filesystem may further be included. The virtual file system creates avirtual BD-ROM (virtual package) by merging an additional content storedon the local storage with a content residing on the BD-ROM, based onmerge management information downloaded with the additional content tothe local storage 113 together. The command interpreter, which is anexecuting entity in the HDMV mode and the BD-J platform, which is anexecuting entity in the BD-J mode can make reference to the virtualpackage in the same manner as the original BD-ROM. During playback ofthe virtual package, the playback apparatus is enabled to executeplayback control, by using data on the BD-ROM and data on the localstorage.

Also, the playback apparatus 200 may further include a networkinterface. A network interface is for performing communication with ancommunication with a device that is external to the playback apparatus.The network interface is capable of accessing a server connected to thenetwork using the Internet, and accessing a server connected by a localnetwork. For example, the network interface can be used to downloadadditional BD-ROM content made available on the Internet, and playingback content using a network function by performing data communicationwith a server specified by the content on the Internet. BD-ROMadditional content refers to content that is not included on an originalBD-ROM, and for example includes additional dubbed audio, subtitles,special features, applications, etc. It is possible to control thenetwork interface from the BD-J platform, and it is possible todownload, to the local storage, additional content made available on theInternet.

(2) There are cases in which the left view and the right view PG streamsare obtained by downloading the streams as additional content. In thiscase, in one AV clip, there are cases in which only left view and rightview PG streams exist. The restriction in such a case is describedbelow.

FIG. 46 is a pattern diagram showing multiplexing of a PG stream (L) anda PG stream (R). Here, the top row shows a structural example of the PGstream (L), and here is composed of two DSs, namely a Display Set (L) 1and a Display Set (L) 2.

The second row from the top shows how a PG stream (L) has been TSpacketized.

The bottom row shows a structural example of a PG stream (R), and hereis composed of two DSs, namely a Display Set (R) 1 and a Display Set (R)2.

The second row from the bottom shows how a PG stream (R) has been TSpacketized.

Here, a pair of DSs made up of a left view and a right view is necessaryfor 3D display. In FIG. 46, the Display Set (L) 1 and the Display Set(R) 1 form a pair, the Display Set (L) 2 and the Display Set (R) 2 forma pair, and these pairs are used for 3D display. When multiplexing, theleft view DS and the right view DS that form a pair are arranged, asshown in FIG. 46, so that the left view DS is always first in thestream. That is to say, the left view DS is provided with a smaller TSpacket number. In addition, the entry table is set for the left viewstream. Accordingly, even during interrupt playback, since the rightview DS can be decoded immediately after the left view DS, the left viewDS and the right view DS can always be decoded as a pair. Therefore, asituation can be avoided in which only one of the pair can be decoded,and the other one cannot be decoded.

Also, the positions of entries to be used for interrupt playback andspecial playback are written in FIG. 46, and by providing both DSs inthe pair of left view and right view farther back than the TS packetthat is indicated by the entry, even when performing interrupt playbackbased on the entry, the paired left view and right view DSs arecorrectly displayed.

Note that although an example is described of providing the DSs as apair after the entry, both DSs may be provided in front of the entry. Inthis case, when performing interrupt playback based on the entry, sinceboth paired DSs cannot be decoded, it is possible to prevent a case inwhich only one only one of the left view and the right view aredisplayed, and thus to reduce discomfort in the viewer.

(3) Although a case is described in the above embodiments of preparingseparate PG streams for left view and right view, here, a first methodis described of adapting one PG stream to 3D display. FIG. 47 shows adata structure of 3D-PCS. The difference between the PCS depicted inFIG. 14 and the 3D-PCS is in the field indicated by composition_object() and an L_R_flag has been added to indicate whether the object is forthe left eye or the right eye. For example, when a composition_object( )indicates that the object is for the left eye, the L_R_flag is set to 0,and when the composition_object( ) indicates that the object is for theright eye, the L_R_flag is set to 1.

Normally when a PG is displayed in 3D, an Object for the left eye andthe Object for the right eye must be a pair. In other words, the valueof the number_of_composition in the 3D-PCS must necessarily be an evennumber, and there must be the same number of composition object ( ) forthe left eye and composition object ( ) for the right eye.

Also, when the 3D-PCS as a functional segment is multiplexed (includinga 2D PCS), as shown in FIG. 48A, the 3D-PCS is provided in a positionahead of the 2D PCS in the stream. In other words, the 3D-PCS isprovided with a smaller TS packet number than the 2D PCS, andaccordingly, since a 3D-compliant playback apparatus can find the 3D-PCSfirst, the 3D-compliant playback apparatus can skip over the 2D PCS thatis not necessary for 3D display.

Also, the 3D-ODS labeled as such in the drawing indicates an ODSincluding an Object that can only be referenced from 3D-PCS. Settingdifferent values for the segment_type of the 3D-ODS and the segment_typeof the ODS enables reducing the decoding load in a playback apparatusthat is not 3D compatible, since the 3D-ODS can be skipped over.Furthermore, when an Object is referenced by both a 2D PCS and a 3D-PCS,defining it as ODS instead of 3D-ODS enables the Object to be sharedbetween 2D and 3D.

Also, for example, when there are a plurality of composition_objects inthe 3D-PCS, corresponding right eye composition_objects ( ) may bearranged to follow the left eye composition_objects ( ) Successivelyarranging such L and R composition_object pairs has the benefit ofenabling easily verifying that the forced_on_flags of respective L and Rcomposition_objects ( ) are the same. Also, restricting the order ofarranging L and R in this way, since L-R flags that indicate whether thecomposition_objects ( ) are L or R can be deleted, enables using exactlythe same data structure as the 2D PCS that already exists, andinterpreting the 3D-PCS with the 2D PCS interpretation program nearlyunchanged.

The following describes a second method for making one PG subtitlestream 3D compatible. The difference from FIG. 48 a is that the left eye3D-PCS (3D-PCS(L)) and the right eye 3D-PCS (3D-PCS (R)) are preparedseparately as a 3D-PCS (L) and a 3D-PCS (R) (see FIG. 48B). A benefit ofthus separating the left eye 3D-PCS and the right eye 3D-PCS in advanceis described next. For 3D-PCSs that include information for both L and Ras described above, it is necessary to set a new flag (L_R_flag) forindicating whether the Object to be referenced is for the left eye orfor the right eye. Also, when a flag is not provided, it is necessary toprovide arrangement restrictions such as specifying that an odd-numberedcomposition_object ( ) is for L and an even numbered composition_object( ) is for R. In other words, when decoding the 3D-PCS, it is necessaryto change the program that interprets the 2D PCS so that a new fieldcalled L_R_flag can be read, or to perform processing so thatodd-numbered composition_objects ( ) and even-numberedcomposition_objects ( ) can be treated respectively as L and R.

On the other hand, since separating the segments as 3D-PCS (L) and3D-PCS (R) and performing transmission enables the structures of 3D-PCS(L) and 3D-PCS (R) to be the same as the structure of the 2D PCS, thesame program can be used as the program used to interpret the 2D PCS. Inthis case, since the decoder distinguishes between the 3D PCSs (3D-PCS(L) and 3D-PCS (R)) and the 2D PCS, the respective segment_types may beset differently. For example, if the segment_type of a normal 2D PCS isset as 0x00, the value of the 3D-PCS (L) segment_type is set as 0x01,and the value of the 3D-PCS (R) segment_type is set as 0x02.Alternatively, by determining the arrangement of the 3D-PCS (L) and the3D-PCS (R) in advance so that the TS packet number of the 3D-pCS (L) issmaller, enables distinguishing between 2D PCSs and 3D-PCSs even whenthe segment_types of both the 3D-PCS (L) and the 3D-PCS (R) are set to0x01.

Also, regarding the PCS arrangement of the 3D-PCS (L), the 3D-PCS (R),and the 2D PCS, providing the 3D-PCS (L) and the 3D-PCS (R) first, andproviding the 2D PCS thereafter, enables reducing the decoding load,since if the 3D-PCS is found first in a 3D-compatible playbackapparatus, the 2D PCS found next can be skipped over.

(4) Although in the above embodiment, both the DSs corresponding to leftview and right view are set with an equal value of composition_number, anew field (for example, L_R_Pairing_number) may be provided, and thisfield may be set with an equal value in both the corresponding DSs.

(5) Although in the above embodiment, the left view PG stream referencesthe object in the left view PG stream, and the right view PG streamreferences the object in the right view PG stream, for example, theright view PG stream may be configured to reference the object of theleft view PG stream. Configuring the left and right PG streams toreference each other's objects has the advantage that there is no needto transmit the same object to right and left, thus enabling conservingbandwidth necessary for transmission.

(6) Although in the above embodiment, the liquid crystal shutter goggles104 are composed of a pair of liquid crystal shutters and a control unitand present stereoscopic view to the user with the use of the parallaxbetween the user's eyes, the present invention is not limited to this.For example, there are glasses on the market that realize stereoscopicview by using a polarization filter instead of liquid crystal, andglasses using this type of mechanism may also be used.

(7) Although in the above embodiment, the description is premised on themethod of causing stereoscopic viewing with use of the liquid crystalgoggles 104, another method may also be used for showing a left viewimage and a right view image to the left eye and the right eye,respectively. For example, a method that does not use special viewingequipment such as glasses, that uses a side-by-side method, or a displaysuch as a lenticular lens, may be used.

Furthermore, regarding the following technological topics, furtherimprovements and modifications may be added. Whether to implement theembodiments in the manner described, or to use any of theimprovements/modifications, is at the arbitrary discretion of the oneimplementing the invention.

Implementation as Recording Apparatus

When the playback apparatus 200 includes a local storage and a removablemedium drive, since it is assumed in this structure that the writingwill be performed to the built-in medium drive and the removable medium,the playback apparatus described in the present specification can besaid to have a dual function as a recording apparatus. When the playbackapparatus 200 functions as a recording apparatus, the writing ofmanagement objects is performed according to the following two modes.

i) When the playback apparatus 200 has a function of executing playbackof a virtual package, the BD-J object is written as follows. That is,when the BD-ROM is inserted, according to a request from an application,additional content corresponding to the BD-ROM is acquired from a WWWserver via a network. The acquired additional content includes the BD-Jobject in which the GUI management table is written. The “GUI managementtable” is a management table used when an application in operationperforms GUI, and includes a resolution to use when executing GUIdisplay, font data to use for the GUI, and a mask flag that specifies,when a user performs a menu call or a title call on the GUI, whether tomask such calls.

In the playback apparatus 200, a control unit that performs recordingcontrol writes the acquired BD-J object to the local storage accordingto a request from the application. By doing this, the contents recordedon the BD-ROM are combined with the additional contents recorded in thelocal storage to constitute the virtual package.

Here, the BD-ROM has recorded thereon an identifier of a disk rootcertificate, an identifier of an organization to which BD-ROM contentshave been distributed, and a identifier of the BD-ROM. An area in whichthe additional contents are to be stored is specified by a file pathincluding the disk root certificate identifier, the organizationidentifier, and the BD-ROM identifier.

The application performs the writing by passing to the control unit thefile path that specifies the area in which the additional content is tobe stored.

When the local storage has a file system with restrictions on the namesof directories and files to be 255 characters or less, the file pathused for writing to the local storage includes file names and extensionsin a 8.3-format file system in which a directory name is limited to 8characters or less and a file name as well as an extension name islimited to three characters or less.

ii) When the playback apparatus 200 supports the function of receivingan on-demand manufacturing service or an electronic sell-through service(MODEST), the BD-J object is written in the following manner.

When the playback apparatus 200 receives the BD-J object supplied viathe on-demand manufacturing service or the electronic sell-throughservice, a default directory and a MODEST directory are created belowthe root directory of the removable medium, and a BDMV directory iscrated below the MODEST directory. The MODEST directory is a firstMODEST directory, which is the MODEST directory that is created when theservice is received for the first time. At the time of receiving theservice for the second time onward, the control unit of the playbackapparatus 200 creates another MODEST directory corresponding to eachservice.

As described above, upon acquiring the BD-J object in which the GUImanagement table is written, the control unit writes a start-up programto the default directory, and writes the BD-J object to the BDMVdirectory below the MODEST directory. This start-up program is a firstprogram to be executed when the recording medium is loaded in therecording apparatus 200. The start-up program causes the playbackapparatus 200 to display a menu allowing a user to make an operation forselecting a BDMV directory. In response to such a user operation, theplayback apparatus 102 executes a root change function. In response to auser operation made on the menu, the root change function causes theplayback apparatus to recognize as the root directory the MODESTdirectory to which the selected BDMV directory belongs. The root changefunction enables executing playback control, by the same controlprocedure as playing back the BD-ROM, based on the acquired BD-J object.

Java™ Application

The BD-J application may be, for example, an Electronic Commerce (EC)client application, or may be an online game played against opponents onthe Internet. Furthermore, by working together with a search engine,various online services can be provided to the user.

Units Incorporating the GUI Management Table

The GUI management table may be provided in a BD-J object. Also, the GUImanagement table may be configured so that PlayList information andPlayItem information are in correspondence. At a timing when a currentPlayList becomes a specified PlayList or a current PlayItem becomes aspecified PlayItem, after releasing the plane memory, a plane memory maybe reserved for stereoscopic playback or a plane memory may be reservedfor monoscopic playback. By doing this, the area management of thememory device is performed with greater time precision.

Video Streams for Stereoscopic View

In the above description, a pair of left- and right-view video streamsare recorded on a BD-ROM but this is only one example. An EnhancedViewvideo stream recorded on a BD-ROM may be a video stream with informationindicating the depth value of per pixel for each of a plurality ofpictures and playback of such a video stream is executed.

Packages to be Implemented

When implementing a playback apparatus, the following BD-J Extensionsare preferably implemented in the playback apparatus. The BD-JExtensions include various packages that have been specialized forproviding functions exceeding GEM [1.0.2] to a Java™ format. Packages towhich BD-J Extensions are provided are as follows.

-   -   org.bluray.media

This package provides specialized functions to be added to the Java™Media FrameWork. Control for angles, audio, and subtitle selection areadded to the package.

-   -   org.bluray.ti

This package includes a structure for referring to an API for operatingby mapping “services” to “titles” according to GEM [1.0.2] and referringto title information from a BD-ROM, and a structure for selecting a newtitle.

-   -   org.bluray.application

This package includes an API for managing the life cycles ofapplications. Also, this package includes an API for referring tonecessary information for signaling when executing applications.

-   -   org.bluray.ui

This package includes a class that defines constant numbers for keyevents specialized in the BD-ROM, and realizes synchronization with thevideo playback.

-   -   org.bluray.vfs

This package provides a binding scheme for binding contents that arerecorded on the BD-ROM (on-disk contents) and contents in the LocalStorage that are not recorded on the BD-ROM (off-disk contents), so thatseamless playback of the contents are enabled regardless of the locationof the contents.

The Binding Scheme associates contents residing on the BD-ROM (AV clips,subtitles, BD-J applications) with related contents residing on theLocal Storage. This Binding Scheme realizes seamless playback regardlessof the location of the contents.

Application Range of Programming Language

It is described in the above embodiment that Java™ language is used asthe programming language for the virtual machine. However, theprogramming language is not limited to Java™, and other programminglanguages, such as B-Shell, Perl Script, and ECMA Script, which are usedin UNIX™ OS and the like, may also be used.

Changing to Multidrive

The above-described embodiment describes a BD-ROM as an example of arecording medium, and a BD-ROM drive as an example of a specific devicehaving a function of reading data from the BD-ROM. However, a BD-ROM ismerely one example, and it is also possible to perform the operationsdescribed in the above embodiment when an optical disk medium such asBD-R, BD-RE, DVD, or CD is used as the recording medium, data having theabove-described data structure is stored on such recording medium, andthere is a drive device capable of reading such recording medium.

The recording media of the embodiments include all types of packagemedia such as optical disks, semi-conductor memory cards, etc. Therecording media of the above embodiment is described by way of anexample related to an optical disk (for example, an existing read-onlyoptical disk such as a BD-ROM or a DVD-ROM). However, the presentinvention is not limited to this. For example, it is possible that aterminal device having a function of writing 3D contents (for example,the function may be included in the playback apparatus, or may beincluded in an apparatus other than the playback apparatus) writes 3Dcontents, including data that is necessary for implementing the presentinvention and has been broadcast or distributed over a network, on awritable optical disk (for example, an existing writable optical disksuch as BD-RE or DVD-RAM). The present invention may be implanted byexecuting playback of such a recording medium by the playback apparatusaccording to the present invention.

Also, it is possible to implement the present invention when therecording medium is, besides an optical disk, for example, a removablemedium such as an SD memory card (semiconductor memory card).

When a semiconductor memory is used instead of a BD-ROM, a structure maybe used in which data is transferred to the read buffer 2, the heapmemory 207, the interactive scenario memory 209, and the static memory205, via an interface for reading data stored in the semiconductormemory card.

More specifically, the following structure may be used. That is, when asemiconductor memory card is inserted in a slot (not depicted) in theplayback apparatus 200, the playback apparatus 200 and the semiconductormemory card are electrically connected via the memory card I/F. The datarecorded on the semiconductor memory card may be transferred via thememory card I/F, to the read buffer 2, the heap memory 207, theinteractive scenario memory 209, and the static scenario memory 205.

The following describes the mechanism of copyright protection of datastored on the BD-ROM. From a standpoint, for example, of improving theconfidentiality of data and copyright protection, there are cases inwhich portions of the data recorded on the BD-ROM are encoded asnecessary.

For example, among the data recorded on the BD-ROM, encryption may beapplied to data corresponding to a video stream, data corresponding toan audio stream, and/or data corresponding to a stream that includesboth video and audio.

The following describes decrypting encoded data that is among the datarecorded on the BD-ROM.

Data corresponding to a key necessary for decrypting encoded data on theBD-ROM (for example, a device key) is stored in advance in the playbackapparatus.

On the other hand, the BD-ROM has recorded thereon data corresponding tothe key necessary for decrypting encoded data (for example, an MKB(media key block) corresponding to the device key), and data that is thekey for deciphering encoded data, that is itself encoded (for example anencoded title key corresponding to the device key and the MKB), isrecorded. Here, the device key, the MKB, and the encoded title keycorrespond to each other, and furthermore correspond to an identifier(for example, a volume ID) written in an area that cannot be normallycopied on the BD-ROM (an area called BCA). If this combination is notcorrect, the code cannot be deciphered. Only if the combination iscorrect, the key necessary for deciphering the code (for example, adecoded title key obtained by decoding the encoded title key based onthe device key, the MKB and volume key, can be elicited, and with use ofthe key necessary for the encoding, the encoded data can be deciphered.

When the inserted BD-ROM is played back in the playback apparatus,encoded data cannot be played back unless the BD-ROM includes a devicekey that is paired with a title key or MKB (or corresponds to a titlekey or MKB). The reason is that the key necessary for deciphering theencoded data (the title key) is itself encoded when recorded on theBD-ROM (as an encoded title key), and if the combination of the MKB andthe device key is not correct, the key necessary for deciphering thecode cannot be elicited.

On the other hand, the playback apparatus is configured so that, if thecombination of the encoded title key, MKB, device key, and volume ID iscorrect, the video stream is decoded, for example with use of the keynecessary for deciphering the code (the decoded title key obtained bydecoding the encoded title key based on the device key, the MKB and thevolume ID), and the audio stream is decoded by the audio decoder.

Although the above description pertains to the mechanism of copyrightprotection of data stored on a BD-ROM, the recording medium is notlimited to being a BD-ROM, and the present invention can be implementedeven when using, for example, a readable/writable semiconductor memory(for example, a semiconductor memory card having a nonvolatile propertysuch as an SD card).

For example, the playback apparatus may be configured to record datacorresponding to data recorded on the BD-ROM on a memory card with useof digital distribution, and to playback the data from the semiconductormemory card. When distributing the necessary data with use of digitaldistribution and recording the distributed data, it is preferable todistribute the data after having performed partial or entire decoding ofthe distributed data as necessary, and leaving data that is necessaryfor the semiconductor memory card in an encoded state.

The following describes operation using digital distribution, forrecording data (distributed data) corresponding to the data described inthe above embodiments on the semiconductor memory.

The operations described above may be configured to be performed by theplayback apparatus described in the embodiments, or by a terminalapparatus dedicated to recording distributed data on a semiconductormemory that is separate from the playback apparatus in the embodiments.Here, an example of the playback apparatus performing the operations isdescribed. Also, an SD card is described as an example of the recordingdestination.

When recording distributed data to the SD memory card inserted in theslot of the playback apparatus, first, transmission is requested of thedistributed data to a distribution server (not depicted) thataccumulates the distributed data. The playback apparatus reads, from theSD memory card, information for uniquely identifying the SD memory cardthat is inserted at the playback apparatus at this time (for example, aspecific identification number assigned individually to the particularSD memory card, more specifically, a serial number of the SD memorycard, etc.), and transmits the read identification information to thedistribution server along with the distribution request.

This identification information for uniquely identifying the SD memorycard corresponds to, for example, the above-described volume ID.

Meanwhile, in the distribution server, decoding is performed so thatnecessary data among the data that is distributed (video streams, audiostreams, etc.) can be deciphered with use of the key that is necessaryfor deciphering the code (for example, the title key), and the necessarydata is stored in the server.

For example, a private key is stored in the distribution server, and thedistribution server is configured so that different public keys areinteractively created to correspond respectively to the semiconductormemory card-specific identification numbers.

Also, the distribution server is configured so that encoding is possibletowards the key that is necessary for deciphering the encoded dataitself (the title key) (in other words, configured so that an encodedtitle key can be generated).

The generated public key information includes information correspondingto the above-described MKB, volume ID, and encoded title key. If thecombination of, for example, the semiconductor memory card-specificidentification number, the actual public key included in the public keyinformation described later, and the device key recorded in advance inthe recording apparatus, is correct, the key necessary for decipheringthe code (for example, the title key obtained by decoding the encodedtitle key based on, for example, the device key, the MKB, and thesemiconductor memory card-specific identification number) is acquired,and with use of this acquired key (title key) necessary for decipheringthe code, decoding of the encoded data can be performed.

Next, the playback apparatus records the received public key informationand distributed data in the recording area of the semiconductor memorycard inserted in the slot.

Next, the following describes an exemplary method for decoding andplaying back encoded data, from among the data included in the publickey information recorded in the recording area of the semiconductormemory card and the data included in the distribution data.

The received public key information is, for example, recorded on adevice list indicating the public key itself (for example, the MKB andthe encoded title key), signature information, the semiconductor memorycard-specific identification number, and information pertaining to adevice to be invalidated.

The signature information includes, for example, hash values of thepublic key information.

In the device list, information is recorded pertaining to, for example,an apparatus that is possibly performing unauthorized playback. This isfor example a device key, an identification number of the playbackapparatus, or an identification number of a decoder in the playbackapparatus recorded in advance on the playback apparatus, information foruniquely specifying the device, a part included in the apparatus, or afunction (program) of the apparatus possibly performing unauthorizedplayback.

The following description pertains to playback of encoded data fromamong the distribution data recorded in the recording area of thesemiconductor memory card.

First, a check is performed pertaining to whether the encoded key itselfmay be operated before decoding the encoded data with use of the publickey itself.

Specifically, the following checks are performed:

(1) whether there is a match between the semiconductor memoryidentification information included in the public key information andthe specific identification number stored in advance on thesemiconductor memory card,

(2) whether there is a match between a hash value of the public keyinformation calculated in the playback apparatus, and a hash valueincluded in the signature information, and

(3) whether, based on information indicated in the device list includedin the public key information, the playback apparatus performingplayback is possibly performing unauthorized playback (for example, bychecking whether the device key included in the device list matches thedevice key stored in the playback apparatus in advance). These checksmay be performed in any order.

Control is performed so that the playback apparatus does not decode theencoded data if any of the following is satisfied, in theabove-described checks 1 to 3: i) the semiconductor memory-specificidentification information included in the public key information doesnot match the specific identification number stored in advance on thesemiconductor memory card, ii) the hash value of the public keyinformation calculated in the playback apparatus does not match the hashvalue included in the signature information, or iii) a judgment is madethat the playback apparatus performing the playback is possiblyperforming unauthorized playback.

Also, a judgment is made that the combination of the semiconductormemory-specific identification number, the public key included in thepublic key information, and the device key recorded in advance in theplayback apparatus, is correct if i) the semiconductor memorycard-specific identification information included in the public keyinformation matches the specific identification number saved on thesemiconductor memory card in advance, (ii), the hash value of the publickey information calculated in the playback apparatus matches the hashvalue included in the signature information, and (iii) a judgment ismade that the playback apparatus is not possibly performing unauthorizedplayback. When the combination is judged to be correct, the encoded datais deciphered with use of the key necessary for deciphering the code(based on a device key, the MKB and the semiconductor memory-specificidentification number).

For example, when the encoded data is a video stream and an audiostream, the video decoder decodes (decodes) the video stream with use ofthe key necessary for deciphering the code (the title key obtained bydecoding the encoded title key), and the audio decoder decodes (decodes)the audio stream with use of the key necessary for deciphering the code.

According to this type of structure, for any playback apparatus, parts,function (program), etc. that is possibly performing unauthorized use atthe time of electronic distribution, information for the identificationof such is provided to the device list, and if distribution isattempted, since playback decoding with use of public key information(the public key) can be suppressed on the playback apparatus side ifinformation is included that is indicated in the device list, even ifthe combination of the semiconductor memory-specific identificationnumber, the public key included in the public key information, and thedevice key recorded in the playback apparatus in advance, is correct,since control can be performed so that the deciphering of the encodeddata is not performed, use of distributed data on an unauthorized devicecan be suppressed.

Also, it is preferable to use a structure in which the semiconductormemory card-specific identifier recorded in advance on the semiconductormemory card is stored in a highly confidential recording area. Thereason is that when the specific number recorded on the semiconductormemory card (for example, in the example of an SD memory card, the SDmemory card serial number, etc.) has been altered, illegal copying isfacilitated. The reason is that different specific identificationnumbers are allocated to different semiconductor memory cards, but ifthe specific identification numbers are altered to be the same, thejudgment in (1) becomes meaningless, and there is a possibility ofillegal copying, corresponding to the number that was altered, beingperformed.

Accordingly, it is preferable for the information that is thesemiconductor memory card-specific identification number to be recordedin a high-confidentiality recording area.

To realize this type of structure, for example by providing a recordingarea (called a second recording area) that is separate from therecording area (called a first recording area) that stores normal dataas recording areas for recording high-confidentiality data that issemiconductor memory card-specific identifiers), and providing a controlcircuit for accessing the recording area, access to the second recordingarea can be made only via the control circuit.

For example, the data recorded in the second recording area has beenencoded and recorded. For example, a circuit for decoding the encodeddata is built into the control circuit. When there is access of the datain the second recording area via the control circuit, the structure needmerely be such that the code is decoding and the decoded data isreturned. Also, if the control circuit stores information of a storagelocation of data recorded in the second recording area, and there is arequest to access the data, the corresponding storage location of thedata need merely be specified, and the data read from the specifiedstorage location be returned.

Upon issuing an access request to the data recorded in the secondrecording area to the control circuit via the memory card I/F (forexample, semiconductor memory-specific identification number),applications that operate in the playback apparatus that requestrecording to the semiconductor memory card with use of digitaldistribution, the control circuit that received the request reads thedata recorded in the second recording area and returns the data to theapplication operating in the playback apparatus. Along with thesemiconductor memory card-specific identification number, thedistribution request for the necessary data need only be requested fromthe distribution server, and the public key information sent from thedistribution server and the corresponding request for distribution ofthe data, may be recorded to the first recording area.

Also, an application operating in the playback apparatus, that requestsrecording on the semiconductor memory card with use of digitaldistribution, before issuing the request to the control circuit via thememory card I/F to access the data recorded on the second recording area(for example, the semiconductor memory card-specific identificationnumbers), preferably checks in advance whether the application has beenaltered. For example, a digital certificate compliant with preexistingX.509 specifications may be used in the check for alteration.

Also, access to the distribution data recorded in the first recordingarea of the semiconductor memory card need not necessarily be access viaa control circuit on the semiconductor memory card.

Implementation as a Program

The application program described in the embodiments can be made asdescribed below. First, the software developer, with use of aprogramming language, writes a source program to realize the content ofthe flowcharts and the functional structural elements. When writing thesource program that embodies the content of the flowcharts and thefunctional structural elements, the software developer uses the classstructures, variables, array variables, and external function calls towrite the program in accordance with the syntax of the programminglanguage.

The written source programs are given as files to a compiler. Thecompiler translates the source programs and creates an object program.

The translation by the compiler is made up of the processes of syntaxanalysis, optimization, resource allocation, and code generation. Syntaxanalysis involves performing lexical analysis and semantic analysis ofthe source programs, and converting the source programs to anintermediary program. Optimization involves performing operations todivide the intermediary program into basic blocks, analyze the controlflow of the intermediary program, and analyze the data flow of theintermediary program. In resource allocation, to improve suitabilitywith a command set of a targeted processor, variables in theintermediary program are allocated to a register or a memory in atargeted processor. Code generation is performed by converting theintermediary commands in the intermediary program into program code, andobtaining an object program.

The object program generated here is made up of one or more programcodes for executing, on a computer, the steps of the flowcharts and thevarious processes carried out by the functional structural elements inthe embodiments. Here, program code may be any of various types such asnative code of a processor or JAVA byte code. There are various formatsfor realization of the steps by the program code. If it is possible touse external functions to realize the steps, call statements that callsuch functions become program code. Also, there are cases in which aprogram code for realizing one step is attributed to separate objectprograms. Ina RISC processor in which command types are limited, thesteps of the flowcharts may be realized by combining calculationoperation commands, logical calculation commands, branch instructioncommands, etc.

When the object programs have been created, the programmer starts up alinker. The linker allocates the object programs and library programs tomemory spaces, combines the object programs and library programs intoone, and generates a load module. The load module generated thus isanticipated to be read by a computer, and causes the computer to executethe processing procedures and functional structural components shown inthe flowcharts. The programs may be provided to users by being recordedon a recording medium that is readable by a computer.

Singular Implementation of the LSI

The system LSI is obtained by implementing a bare chip on a high-densitysubstrate and packaging them. The system LSI is also obtained byimplementing a plurality of bare chips on a high-density substrate andpackaging them, so that the plurality of bare chips have an outerappearance of one LSI (such a system LSI is called a multi-chip module).

The system LSI has a QFP (Quad Planar view Package) type and a PGA (PinGrid Array) type. In the QFP-type system LSI, pins are attached to thefour sides of the package. In the PGA-type system LSI, a lot of pins areattached to the entire bottom.

These pins function as an interface with other circuits. The system LSI,which is connected with other circuits through such pins as aninterface, plays a role as the core of the playback apparatus 200.

Such a system LSI can be embedded into various types of devices that canplay back images, such as a television, game machine, personal computer,one-segment mobile phone, as well as into the playback apparatus 200.The system LSI thus greatly broadens the use of the present invention.

As shown in the above embodiment, when a buffer, video decoder, audiodecoder, and graphics decoder are integrated into a system LSI, it isdesirable that the system LSI conforms to the Uniphier architecture.

A system LSI conforming to the Uniphier architecture includes thefollowing circuit blocks.

-   -   Data Parallel Processor (DPP)

The DPP is an SIMD-type processor where a plurality of elementalprocessors perform a same operation. The DPP achieves a paralleldecoding of a plurality of pixels constituting a picture by causingoperating units, respectively embedded in the elemental processors, tooperate simultaneously by one instruction.

-   -   Instruction Parallel Processor (IPP)

The IPP includes: a local memory controller that is composed ofinstruction RAM, instruction cache, data RAM, and data cache; processingunit that is composed of instruction fetch unit, decoder, executionunit, and register file; and virtual multi processing unit that causesthe processing unit to execute a parallel execution of a plurality ofapplications.

-   -   MPU Block

The MPU block is composed of: peripheral circuits such as ARM core,external bus interface (Bus Control Unit: BCU), DMA controller, timer,vector interrupt controller; and peripheral interfaces such as UART,GPIO (General Purpose Input Output), and sync serial interface.

-   -   Stream I/O Block

The stream I/O block performs data input/output with the drive device,hard disk drive device, and SD memory card drive device which areconnected onto the external busses via the USB interface and the ATApacket interface.

-   -   AV I/O Block

The AV I/O block, which is composed of audio input/output, videoinput/output, and OSD controller, performs data input/output with thetelevision and the AV amplifier.

-   -   Memory Control Block

The memory control block performs reading and writing from/to the SD-RAMconnected therewith via the external buses. The memory control block iscomposed of internal bus connection unit for controlling internalconnection between blocks, access control unit for transferring datawith the SD-RAM connected to outside of the system LSI, and accessschedule unit for adjusting requests from the blocks to access theSD-RAM.

The following describes a detailed production procedure. First, acircuit diagram of a part to be the system LSI is drawn, based on thedrawings that show structures of the embodiments. And then theconstituent elements of the target structure are realized using circuitelements, ICs, or LSIs.

As the constituent elements are realized, buses connecting between thecircuit elements, ICs, or LSIs, peripheral circuits, interfaces withexternal entities and the like are defined. Further, the connectionlines, power lines, ground lines, clock signals and the like aredefined. For these definitions, the operation timings of the constituentelements are adjusted by taking into consideration the LSIspecifications, and bandwidths necessary for the constituent elementsare secured. With other necessary adjustments, the circuit diagram iscompleted.

After the circuit diagram is completed, the implementation design isperformed. The implementation design is a work for creating a boardlayout by determining how to arrange the parts (circuit elements, ICs,LSIs) of the circuit and the connection lines onto the board.

After the implementation design is performed and the board layout iscreated, the results of the implementation design are converted into CAMdata, and the CAM data is output to equipment such as an NC (NumericalControl) machine tool. The NC machine tool performs the SoCimplementation or the SiP implementation. The SoC (System on Chip)implementation is a technology for printing a plurality of circuits ontoa chip. The SiP (System in Package) implementation is a technology forpackaging a plurality of circuits by resin or the like. Through theseprocesses, a system LSI of the present invention can be produced basedon the internal structure of the playback apparatus 200 described ineach embodiment above.

It should be noted here that the integrated circuit generated asdescribed above may be called IC, LSI, ultra LSI, super LSI or the like,depending on the level of the integration.

It is also possible to achieve the system LSI by using the FPGA (FieldProgrammable Gate Array). In this case, a lot of logic elements are tobe arranged lattice-like, and vertical and horizontal wires areconnected based on the input/output combinations described in LUT(Look-Up Table), so that the hardware structure described in eachembodiment can be realized. The LUT is stored in the SRAM. Since thecontents of the SRAM are erased when the power is off, when the FPGA isused, it is necessary to define the Config information so as to write,onto the SRAM, the LUT for realizing the hardware structure described ineach embodiment.

In the embodiment, the invention is realized by middleware and hardwarecorresponding to the system LSI, hardware other than the system LSI, aninterface portion corresponding to the middleware, an interface portionto intermediate between the middleware and the system LSI, an interfaceportion to intermediate between the middleware and the necessaryhardware other than the system LSI, and a user interface portion, andwhen integrating these elements to form the playback apparatus,particular functions are provided by operating the respective elementsin tandem.

Appropriately defining the interface corresponding to the middleware andthe interface for the middleware and the system. LSI enables parallel,independent development of the user interface portion, the middlewareportion, and the system LSI portion of the playback apparatusrespectively, and enables more efficient development. Note that thereare various ways of dividing up the respective interface portions. Forexample, when the video decoder, graphics decoder, audio decoder, andadder (composite unit) are fabricated on a single chip, development ofan interface portion between the middleware to control these units andthe middleware corresponding to these units is performed when developingthe chip. After completion, including the developed middleware andinterface portion in a storage unit such as a memory of the playbackapparatus, along with integrating the chip into the playback apparatus,enables performing development of the playback apparatus and the chip inparallel, thereby improving development efficiency.

Versatility is improved when the same interface portion is usedregardless of the type of developed chip and the middleware pertainingto the developed chip.

Needless to say, the portion structured as a system LSI in the abovedescription is not limited to being structured as an LSI, and mayinstead be configured with use of a signal processing circuit thatincludes corresponding functions to those to be included in the systemLSI.

Recording Medium

The following describes a method of making the recording medium, inother words, a use mode of the recording method for the recording mediumon which predetermined data is recorded.

Recording methods pertaining to the above embodiment include not onlyreal time recording in which AV clips and other data are created in realtime and written directly to an area on the recording medium, but also amethod in which optical disks are mass-produced by creating an entireimage of a bit stream to be recorded in a volume area in advance,creating a master disk based on this bit stream, and pressing this disk,namely, pre-format recording. The recording medium pertaining to theabove embodiment may also be specified by either a recording method byreal time recording, or a recording method by pre-formatting recording.

These recording methods include the following steps: importing datamaterials such as video, audio, subtitles, and menus; obtaining a PESstream by digitally converting and compression coding imported datamaterials and encoding according to MPEG; obtaining an AV clip bymultiplexing the PES stream; creating AV files and non-AV files in aBD-ROM-compliant file format by inputting a Java™ program and AV filesobtained in the multiplexing step, writing non-AV files on the recordingmedium, and writing AV files on the recording medium.

Positioning Extents in the Volume Area

It has already been explained that when the predetermined data is storedin a file and recorded on the recording medium, the predetermined datais handled in the form of “extents” in the file system. In the recordingmedium, the addresses of such extents are managed as follows.

Extents are formed in a plurality of physically continuous sectors inpartition spaces in the volume area. Partition spaces refer to areasthat are structured from “areas in which file set descriptors arerecorded”, “areas in which terminal descriptors are recorded,” “ROOTdirectory areas”, “BDMV directory areas”, “JAR directory areas”, “BDJOdirectory areas”, “PLAYLIST directory areas”, “CLIPINF directory areas”,and “STREAM directory areas”, and are areas accessed by the file system.The following describes these areas.

“File set descriptors” include logical block numbers (LBN) pointing towhich sector, of the areas in the directory, has a file entry of theROOT directory recorded therein. “Terminal descriptors” indicate aterminal of the file set descriptor.

Next, the details of the directory areas are described. A plurality ofdirectory areas as described above all have a common structure. That isto say, the “directory area” is composed of a “file entry”, a “directoryfile”, and a “file recording area for lower rank files”.

The “file entry” includes a “descriptor tag”, an “ICB tag”, and an“allocation descriptor”.

The “descriptor tag” is a tag indicating that the file entry is a fileentry.

The “ICB tag” indicates attribute information pertaining to the fileentry.

The “allocation descriptor” includes a logical block number (LBN)indicating a recording position of directory files. This concludes thedescription of file entries. The following describes the details ofdirectory files.

“Directory files” include “file identification descriptor for low-rankdirectory” and “file identification descriptors for low-rank files”.

The “file identification descriptor for low-rank directory” is referenceinformation for accessing a low-rank file directory that is under thefile entry, and includes identification information that indicates thatlow-rank directory, a length of a directory name of the lower rankdirectory, a file entry address that indicates in which logical blocknumber is the low rank directory file entry written, and a directoryname of that low rank directory.

The “file identification descriptor for low-rank files” is referenceinformation for accessing a low-rank file that is under the file entry,and includes identification information indicating the low-rank file, alength of the low-rank file name, a file entry address indicating whichlogical block number stores the file entry therein for the low-rankfile, and a file name of the low-rank file.

Since the file identification descriptors in directory files of thesedirectories indicate in which logical block the low-rank directories andfile entries of the low-rank files are stored, tracking back these fileidentification descriptors enables arriving at a file entry of the BDMVdirectory from the file entry of a ROOT directory, and arriving at afile entry of the PLAYLIST directory from the file entry of the BDMVdirectory. File entries of the JAR directory, the BDJO directory, theCLIPINF directory, and the STREAM directory can be arrived at similarly.

The “file recording area of low-rank files” is an area in which thesubstance of a low-rank file under a predetermined directory is stored.A “file entry” and one or more “extents” are stored in the “filerecording area of low-rank files”.

The “file entry” includes a “descriptor tag”, an “ICB tag”, and an“allocation descriptor”.

The “descriptor tag” is a tag indicating that the file entry is a fileentry. There are types of tags such as file entry descriptors, space bitmap descriptors, etc., and in the case of file entries, “261” is writtenas a descriptor tag to indicate a file entry.

The “ICB tag” indicates attribute information pertaining to the fileentry.

The “allocation descriptor” includes a logical block number (LBN)indicating a recording position of extents making up the low-rank filesunder a certain directory. The allocation descriptor includes dataindicating an extent length, and a logical block number indicating arecording position of the extent. However, setting the upper 2 bits ofdata indicating the extent length to “0” indicates an extent for whichallocation and recording has finished, and setting the upper 2 bits ofdata to “1” indicates an extent for which allocation has finished butrecording has not finished. Setting to “0” indicates that the extent isa continuing extent of the allocation identifier. When a lower rank fileunder a certain directory is divided into a plurality of extents, thefile entry has a plurality of allocation descriptors for each extent.

Referencing an allocation descriptor of a file entry as described aboveenables knowing an address of extents that make up PlayList informationfiles, clip information files, AV clip files, BD-J object files, and JARarchive files.

Files that store streams, data, and flags that are the central focus ofthe present application refer to the file recording areas existing inthe directory area of directories to which the files belong, and can beaccessed by tracking back file identification descriptors in files ofthe directory files, and allocation identifiers in the file entries.

File Opening

The following describes processing when reading files on the recordingmedium to the device.

Since AV streams, Index.bdmv, JAR files, and BD-J objects as describedabove are recorded on the BD-ROM in accordance with the file structureand the directory structure, the playback device can read these to amemory by performing a system call for file opening.

File opening refers to searching directories by file name given at atime of a system call, and if the file exists, reserving an FCB (FileControl Block), and performing processing to return the file handle.

Application of Real-Time Recording

The premise of the previous description is that, in pre-recordingtechnology in an authoring system, AV clips and PlayList information arerecorded on a BD-ROM and provided to a user. However, in real timerecording, the AV clips and PlayList information may be recorded on aBD-RE, BD-R, hard disk, or semiconductor memory card, and provided to auser.

In this case, AV clips may be transport streams obtained by real timeencoding performed by the recording apparatus on an analog input signal,or may be a transport stream obtained by partializing a transport streamthat the recording device has entered digitally.

The recording device that executes real-time recording includes a videoencoder that obtains a video stream by encoding a video signal, an audioencoder that obtains an audio stream by encoding the audio signal, amultiplexer that obtains an MPEG2-TS format digital stream bymultiplexing video streams, audio streams, etc., and a source packetizerthat converts TS packets that make up the MPEG2-TS format digital streaminto source packets, and store the MPEG2 digital streams that have beenconverted to the source packet format in AV clip files, and write thestored digital streams on a BD-RE, a BD-R, etc.

The control unit of the recording device writes the digital streams, andalso performs processing to generate the clip information and thePlayList information in the memory. Specifically, when recordingprocessing is requested by a user, the control unit creates AV clipfiles and clip information files on the BD-RE and BD-R.

If a start position of a GOP in an audio stream is detected from atransport stream input from an external device, or a GOP of a videostream is generated by an encoder, the control unit of the recordingdevice acquires a PTS of an intra-picture positioned at the start ofthis GOP, and a packet number of the source packet that stores the startportion of this GOP, and adds an entry map of the clip information fileby pairing this PTS and packet number, and an EP_PTS entry with anEP_SPN entry. Thereafter, each time the GOP is generated, thecombination of the EP_PTS entry and the EP_SPN entry is added to theentry map of the clip information file. At this time, when the start ofthe GOP is an IDR picture, an is_angle_change flag set to “on” is addedto the pair of EP_PTS entry and EP_SPN entry. When the start of a GOP isnot an IDR picture, an “is_angle_change” flag set to “off” is added tothe pair of EP_PTS entry and EP_SPN entry.

Also, the attribute information of a stream in the clip information fileis set according to an attribute of a stream to be recorded. In thisway, AV clips and clip information are generated and written on a BD-REor BD-R, and via the entry map in the clip information, PlayListinformation defining a playback channel is generated, and written to theBD-RE or BD-R. Executing this type of processing in real time recordingtechnology enables obtaining a hierarchical structure such as AVclips—clip information—PlayList information.

Implementation as Playback Device Realizing Managed Copying

The playback device described in the embodiments may have a function ofwriting digital streams by managed copying.

Managed copying is technology for performing communication with a serverand enabling copy execution only in a state in which authorization hasbeen performed and permission received, when copying digital streams,PlayList information, clip information, and application programsrecorded on a read-only recording medium such as BD-ROM, etc., to areadable/writable recording medium such as another optical disk (BD-R,BD-RE, DVD-R, DVD-RW, DVD-RAM, etc.), hard disk, removable medium (SDmemory card, memory stick, compact flash memory, smart media, multimediacard, etc.). According to this technology, it is possible performcontrol such as restricting the number of times to perform backup,permitting backup only in state in which billing has been performed,etc.

When realizing copying from a BD-ROM to a BD-R or BD-RE, when arecording amount is the same between a copy source and a copydestination in managed copying, it is sufficient to perform theoperation of sequentially copying, from an innermost circumference to anoutermost circumference, bit streams in the BD-ROM that is the copysource.

Transcoding is necessary if copying between different types of medium isexpected. Here, “transcoding” refers to processing by which a digitalstream is made compliant with the application format of the copydestination medium, by converting the format of the digital streamrecorded on the BD-ROM from the MPEG2 transport stream format to theMPEG2 program stream format, or lowering the bit rate allocated to thevideo stream and the audio stream and re-encoding the stream. It isnecessary to acquire AV clips, clip information, and PlayListinformation by performing the above-described real-time recordingtechnology processing for the relevant transcode.

Playback Apparatus Using an on-Demand Manufacturing Service or anElectronic Sell-Through Service (MODEST)

When the playback apparatus 200 supports the function of receiving anon-demand manufacturing service or an electronic sell-through service(MODEST), the BD-J object is written in the following manner.

When the playback apparatus 200 receives the BD-J object supplied viathe on-demand manufacturing service or the electronic sell-throughservice, a default directory and a MODEST directory are created belowthe root directory of the removable medium, and a BDMV directory iscrated below the MODEST directory. The MODEST directory is a firstMODEST directory, which is the MODEST directory that is created when theservice is received for the first time. At the time of receiving theservice for the second time onward, the control unit of the playbackapparatus 200 creates another MODEST directory corresponding to eachservice.

As described above, upon acquiring the BD-J object in which the GUImanagement table is written, the control unit writes a start-up programto the default directory, and writes the BD-J object to the BDMVdirectory below the MODEST directory. This start-up program is a firstprogram to be executed when the recording medium is loaded in therecording apparatus 200. The start-up program causes the playbackapparatus 200 to display a menu allowing a user to make an operation forselecting a BDMV directory. In response to such a user operation, theplayback apparatus 102 executes a root change function. In response to auser operation made on the menu, the root change function causes theplayback apparatus to recognize as the root directory the MODESTdirectory to which the selected BDMV directory belongs. The root changefunction enables executing playback control, by the same controlprocedure as playing back the BD-ROM, based on the acquired BD-J object.

Method of Data Structure Writing

Among data structures in the embodiments, repeating structures in whicha plurality of information pieces having a specific determined form canbe defined by setting an initial value of a control parameter and arepeat condition in a “for” clause.

Also, a condition to be satisfied and a parameter to be set when thecondition is satisfied can be written using an “if” clause, that is anarbitrary data structure that defines specified information when apredetermined condition is satisfied. In this way, since the datastructure described in the embodiment can be written using the grammarof a high-level programming language, the data structure described inthe embodiments passes through a translation process by a compiler, theprocess including syntax analysis, optimization, resource allocation,and code generation, and is recorded on the recording medium in a stateconverted to a computer-readable computer code, similarly to theabove-described program. In this way, the data structure written in thehigh-level programming language is treated as a member variable of anarray type in a class structure, and forms a portion of the program. Inother words, since the data is converted to computer code and recordedon a recording medium, and becomes a member variable of the program thedata structure in the embodiments is substantially the same as theprogram, and is eligible to receive protection as a computer-relatedinvention.

Positioning PlayList Information and Clip Information in the Program

In order to be loaded from a recording medium to a memory and providedfor execution by a computer, a program formatted to perform AV playbackbased on PlayList information includes a plurality of sections in thememory, namely a text section, a data section, a bss section, and astack section.

The text section is composed of data that does not include program codesequences, initial values, or reconfigurations.

The data section includes initial values, and an arrangement of datathat can be reconfigured during execution. Data that is stored in filesand accessed as needed is stored in the data section of the recordingmedium.

The bss section includes an arrangement of data not including initialvalues. Since the text section accesses the data of the bss section byspecifying an address, an area must be reserved for the bss section on aRAM determined during compiling and linking.

The stack section is an area temporarily allotted to a program asnecessary. Local parameters to be used temporarily for realizing theprocessing of the flowchart are stored here.

Initial values are set in the bss section when initializing the program,and necessary areas are reserved in the stack section when initializingthe program.

Since the PlayList information and clip information are converted tocomputer code as described above and recorded on the recording medium,when executing the program, such information is managed as “data that isnot reconfigured” in the text section, or “data stored in files andaccessed as necessary” in the data section. The PlayList information andclip information described in the embodiments are to be constituentelements of the program at the time of executing the program, and do notqualify as simple data presentation.

The present invention may be any combination of the embodiments andsupplementary remarks.

Although the present invention has been fully described by way ofexample with reference to the accompanying drawings, it is to be notedthat various changes and modifications will be apparent to those skilledin the art. Therefore, unless such changes and modifications otherwisedepart from the scope of the present invention, they should beconstructed as being included therein.

INDUSTRIAL APPLICABILITY

The present invention can be applied to a recording medium on which 3Dvideo contents are recorded, and is particularly effective in a case ofrealizing a stereoscopic view of a 3D character by a PG stream of a leftview and a right view.

REFERENCE SIGNS LIST

-   100 BD-ROM-   200 Playback apparatus-   300 Television-   400 Liquid crystal shutter goggles-   500 Remote control-   201 BD drive-   202 HDMI interface-   203 Playback state/setting register set-   204 Non-volatile memory-   205 Static scenario memory-   206 Playback control engine-   207 Heap memory-   208 BD-J platform-   209 Dynamic scenario memory-   210 Mode management module-   211 Command interpreter-   212 UO detection module-   1 Playback unit-   2 Read Buffer-   3 PID Filter-   4 a, 4 b, 4 c, 4 d, 4 e Transport Buffer-   4 f Periphery circuit-   5 a Video Decoder for Left view-   5 b Video Decoder for Right view-   6 a Video Plane (L)-   6 b Video Plane (R)-   7 Switch-   8 a Graphics Decoder for Left view-   8 b Graphics Decoder for Right view-   9 a Graphics Plane for Left view-   9 b Graphics Plane for Right view-   10 a CLUT UNIT for L-   10 b CLUT UNIT for R-   11 a OFFSET UNIT for L-   11 b OFFSET UNIT for R-   12 CLUT output switch-   13 Adder-   14 Audio Decoder

1. A non-transitory recording medium on which is recorded a videostream, a left view graphics stream and a right view graphics stream,wherein the left view graphics stream and the right view graphics streameach include one or more display sets, each display set is a data groupused for displaying one screen worth of one or more graphics objects,the one or more display sets included in the left view graphics streamare in one-to-one correspondence with the one or more display setsincluded in the right view graphics stream, and an identical playbacktime for the video stream on a playback time axis is set in display setsthat correspond with each other, each display set includes stateinformation indicating that the display set is one of (i) all datanecessary for displaying the one screen worth of one or more graphicsobjects, and (ii) a difference from an immediately previous display set,identical content is indicated by the state information included indisplay sets that correspond with each other in the left view graphicsstream and the right view graphics stream, each display set includes aplurality of function segments, the plurality of function segmentsincluding a presentation composition segment and one or more objectdefinition segments, the graphics objects are defined by the objectdefinition segments, the presentation composition segment is afunctional segment that specifies a screen structure with use of thegraphics objects, the state information is included in the presentationcomposition segment, each presentation composition segment includes ascreen structure number, the screen structure number indicates whichnumber display set the presentation composition segment belongs to, andan identical screen structure number is set in presentation compositionsegments included in display sets that correspond with each other in theleft view graphics stream and the right view graphics stream.
 2. Anon-transitory recording medium on which is recorded a video stream, aleft view graphics stream and a right view graphics stream, wherein theleft view graphics stream and the right view graphics stream eachinclude one or more display sets, each display set is a data group usedfor displaying one screen worth of one or more graphics objects, the oneor more display sets included in the left view graphics stream are inone-to-one correspondence with the one or more display sets included inthe right view graphics stream, and an identical playback time for thevideo stream on a playback time axis is set in display sets thatcorrespond with each other, each display set includes state informationindicating that the display set is one of (i) all data necessary fordisplaying the one screen worth of one or more graphics objects, and(ii) a difference from an immediately previous display set, identicalcontent is indicated by the state information included in display setsthat correspond with each other in the left view graphics stream and theright view graphics stream, each display set includes a plurality offunction segments, the plurality of function segments including apresentation composition segment and one or more object definitionsegments, the graphics objects are defined by the object definitionsegments, the presentation composition segment is a functional segmentthat specifies a screen structure with use of the graphics objects, thestate information is included in the presentation composition segment,each presentation composition segment includes a forced display flag,the forced display flag is a flag that specifies whether to forciblycause a playback device to display the graphics objects in a display setto which the presentation composition segment belongs, and identicalcontent is indicated by forced display flags in presentation compositionsegments included in display sets that correspond with each other in theleft view graphics stream and the right view graphics stream.
 3. Anon-transitory recording medium on which is recorded a video stream, aleft view graphics stream and a right view graphics stream, wherein theleft view graphics stream and the right view graphics stream eachinclude one or more display sets, each display set is a data group usedfor displaying one screen worth of one or more graphics objects, the oneor more display sets included in the left view graphics stream are inone-to-one correspondence with the one or more display sets included inthe right view graphics stream, and an identical playback time for thevideo stream on a playback time axis is set in display sets thatcorrespond with each other, each display set includes state informationindicating that the display set is one of (i) all data necessary fordisplaying the one screen worth of one or more graphics objects, and(ii) a difference from an immediately previous display set, identicalcontent is indicated by the state information included in display setsthat correspond with each other in the left view graphics stream and theright view graphics stream, each display set includes a plurality offunction segments, the plurality of function segments including apresentation composition segment and one or more object definitionsegments, the graphics objects are defined by the object definitionsegments, the presentation composition segment is a functional segmentthat specifies a screen structure with use of the graphics objects, thestate information is included in the presentation composition segment,the left view graphics stream and the right view graphics stream arepacketized elementary streams (PESs), the presentation compositionsegment is stored in one or more PES packets, the playback time is addedto the PES packet that stores the presentation composition segment, andthe playback time indicates at what time to execute the display of thegraphics objects in the display set to which the presentationcomposition segment belongs.
 4. A playback apparatus for thenon-transitory recording medium as recited in claim 1, comprising: aread unit operable to read the left view graphics stream and the rightview graphics stream from the non-transitory recording medium; and acontroller operable to specify a left view screen structure based on adisplay set including a plurality of function segments extracted fromthe read left view graphics stream and specify a right view screenstructure based on a display set including a plurality of functionsegments extracted from the read right view graphics stream.
 5. Aplayback method for the non-transitory recording medium as recited inclaim 1, comprising: reading the left view graphics stream and the rightview graphics stream from the non-transitory recording medium;specifying a left view screen structure based on a display set includinga plurality of function segments extracted from the read left viewgraphics stream; and specifying a right view screen structure based on adisplay set including a plurality of function segments extracted fromthe read right view graphics stream.
 6. A playback apparatus for thenon-transitory recording medium as recited in claim 2, comprising: aread unit operable to read the left view graphics stream and the rightview graphics stream from the non-transitory recording medium; and acontroller operable to specify a left view screen structure based on adisplay set including a plurality of function segments extracted fromthe read left view graphics stream and specify a right view screenstructure based on a display set including a plurality of functionsegments extracted from the read right view graphics stream.
 7. Aplayback method for the non-transitory recording medium as recited inclaim 2, comprising: reading the left view graphics stream and the rightview graphics stream from the non-transitory recording medium;specifying a left view screen structure based on a display set includinga plurality of function segments extracted from the read left viewgraphics stream; and specifying a right view screen structure based on adisplay set including a plurality of function segments extracted fromthe read right view graphics stream.
 8. A playback apparatus for thenon-transitory recording medium as recited in claim 3, comprising: aread unit operable to read the left view graphics stream and the rightview graphics stream from the non-transitory recording medium; and acontroller operable to specify a left view screen structure based on adisplay set including a plurality of function segments extracted fromthe read left view graphics stream and specify a right view screenstructure based on a display set including a plurality of functionsegments extracted from the read right view graphics stream.
 9. Aplayback method for the non-transitory recording medium as recited inclaim 3, comprising: reading the left view graphics stream and the rightview graphics stream from the non-transitory recording medium;specifying a left view screen structure based on a display set includinga plurality of function segments extracted from the read left viewgraphics stream; and specifying a right view screen structure based on adisplay set including a plurality of function segments extracted fromthe read right view graphics stream.